-- Logs begin at Fri 2022-07-29 05:14:50 UTC, end at Fri 2022-07-29 07:16:39 UTC. -- Jul 29 06:11:54.083704 dv-cinder-834822-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 29 06:11:54.093353 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 29 06:11:54 2022] *** Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: nodename: dv-cinder-834822-7-iscsi Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: machine: x86_64 Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: clock source: unix Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: pcre jit disabled Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: detected number of CPU cores: 4 Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: current working directory: / Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: detected binary path: /usr/bin/uwsgi-core Jul 29 06:11:54.095341 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: your processes number limit is 47680 Jul 29 06:11:54.095925 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: your memory page size is 4096 bytes Jul 29 06:11:54.095925 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: detected max file descriptor number: 2048 Jul 29 06:11:54.095925 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: lock engine: pthread robust mutexes Jul 29 06:11:54.095925 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: thunder lock: enabled Jul 29 06:11:54.095925 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 29 06:11:54.095925 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 29 06:11:54.134898 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: Python main interpreter initialized at 0x563c3d2fe9c0 Jul 29 06:11:54.134898 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: python threads support enabled Jul 29 06:11:54.134898 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: your server socket listen backlog is limited to 100 connections Jul 29 06:11:54.134898 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: your mercy for graceful operations on workers is 90 seconds Jul 29 06:11:54.135504 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: mapped 403077 bytes (393 KB) for 2 cores Jul 29 06:11:54.135605 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: *** Operational MODE: preforking *** Jul 29 06:11:54.135881 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: *** uWSGI is running in multiple interpreter mode *** Jul 29 06:11:54.135881 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: spawned uWSGI master process (pid: 80151) Jul 29 06:11:54.136075 dv-cinder-834822-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 29 06:11:54.137174 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: spawned uWSGI worker 1 (pid: 80161, cores: 1) Jul 29 06:11:54.137174 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: spawned uWSGI worker 2 (pid: 80162, cores: 1) Jul 29 06:11:54.137174 dv-cinder-834822-7-iscsi devstack@keystone.service[80151]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 29 06:11:59.656941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 06:11:59.656941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: from cryptography.utils import int_from_bytes Jul 29 06:11:59.657756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 06:11:59.657756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: from cryptography.utils import int_from_bytes Jul 29 06:11:59.657895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 06:11:59.657895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: from cryptography.utils import int_from_bytes Jul 29 06:11:59.657895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 06:11:59.657895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: from cryptography.utils import int_from_bytes Jul 29 06:11:59.706682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 06:11:59.706861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 06:11:59.707047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] command line args: [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 06:11:59.707322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 06:11:59.707498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ================================================================================ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 06:11:59.707753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] admin_token = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.707997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_dir = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.708185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.708267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 06:11:59.708368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_source = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.708447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 06:11:59.708558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] debug = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.708673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] command line args: [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 06:11:59.708779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.709031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 06:11:59.709110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ================================================================================ {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 06:11:59.709184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.709307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.709387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] admin_token = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.709562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.709655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] config_dir = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.709788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.710416 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.710492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] config_source = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.710665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] debug = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.710758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.710898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_config_append = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.710977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_dir = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.711983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] list_limit = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] log_config_append = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] log_dir = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] log_file = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.712940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.713920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.714939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.715174 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.715265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.715391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] publish_errors = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.715487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.715626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.715766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.715881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] standard_threads = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] publish_errors = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.716980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.717059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.717203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.719637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.719712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.719773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.719830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_journal = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.719830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_json = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.719989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_stderr = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_syslog = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] standard_threads = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.720471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.720631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.720757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] use_journal = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] use_json = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.720919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] use_stderr = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.720984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] use_syslog = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.721099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 06:11:59.721216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721466 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.721947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.722916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723080 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723309 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.723908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.724872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.725997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.726117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.726241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.726364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.726485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.726992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.727997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.728987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.729960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.730960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.731957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.732889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.733942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.734065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.734179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.734290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.734401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.734965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.735964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.736938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.737922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.738971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.739924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.740884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.741086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.741177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.741334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.741579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.741668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.741812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.741941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742080 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742551 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.external = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.742906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743303 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.password = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.token = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.743842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] auth.external = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.744987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] auth.password = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.745942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] auth.token = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.746917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.747902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] credential.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.748969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749466 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.749943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.750917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.751973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.752966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.753998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.754867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.755965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.756050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.756199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.756276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.756444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.756522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.757237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.757325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.757506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.757745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.757823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.757985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.758925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.759976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.760997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.761916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.762863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.763962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.764940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.765993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.766937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.767861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.768998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.769950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.770897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.771968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.772123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.772201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.772973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.773972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.774944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.775931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776083 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.776893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777131 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.777830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.778969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.779923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] role.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] role.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.780922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.781901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.782874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.783967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.784974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785906 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.785984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.786926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.787928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.788081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.788333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.788410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.788482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.788575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.788794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.788872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.789852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.790994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.791936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.792997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.connection = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.793956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.794971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 06:11:59.795051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.795279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.795529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.795770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.796022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.796263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.796513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.796772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.797017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.797256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 06:11:59.797430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80162) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 06:11:59.824580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.824815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.825042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.825965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.826203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.826435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.833524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.835642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.863435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.863661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.864938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.865180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.915990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.916903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.966733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.966965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.967838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:11:59.968080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.031246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.031475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.031701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.031884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.031986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.032121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.032205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.032333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.032413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.032580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.032808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.033028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.055651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.055879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.056962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.057042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.057181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.057400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.057619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.067918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.068467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.080759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.080984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.081934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.082062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.082144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.082326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.082542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80162) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 06:12:00.099371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.099577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 06:12:00.099853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.099936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.100067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 06:12:00.100150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 06:12:00.100340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.100479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.101186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 06:12:00.101379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.101486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.101658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.101741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.101919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.101999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.102185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.111540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.111815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.111898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.112088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.112170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.112368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.112450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.112726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.119452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.119689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.124091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.124368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.124450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.124668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.124753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.124945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.125028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.125220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.125301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.125474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.125564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.125737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.125818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.126869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.127058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.150625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.150775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.152118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.152243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.152400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.152522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.157054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.157147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.157339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.157423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.157624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.157706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.164842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.164942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.165130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.165230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.169330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.169421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.169574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.169657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.169862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.169943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.170141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.170221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.170414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.170497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.170700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.170787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.170977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.171060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.171250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.171339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.187011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.187112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.187288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.187372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.191181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.191268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.194449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.194571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.194743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.194884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.199087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.199175 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.202236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.202429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.205642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.205895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.205979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.206197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.206279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.206474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.206568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.206760 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.206842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.207039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.207120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.207318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.223462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.223741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.223823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.224008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.224103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.224275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.224358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.224573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.231661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.232180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.233208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.233483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.233762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.234040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.236259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.236570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.236852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.237075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.237155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.237229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.237399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.237485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.237710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.237994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.248736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.249013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.249290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.249568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.249734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.249835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.250018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.250101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.250302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.250384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.250587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.250871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.251149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.251439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.265964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.267095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.268910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.270082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.271804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.273615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.274956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.276089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.276486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.276795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.277074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.277727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.278029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.278310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.283476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.284775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.285092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.286366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.288611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.288902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.289182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.289453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.290122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.290416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.290704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.290992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.297421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.297708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.297984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.299019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.299308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.299589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.304817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.305095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.305383 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.305655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.305933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.306210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.306481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.306761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.306890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80162) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 06:12:00.307127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.307207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.307411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.307490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.307693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.307772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.307954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.308043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.308227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.308310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.308485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.308585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.308790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.309171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.309457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.309743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.310021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.310293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.310557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80162) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 06:12:00.342922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563c3d2fe9c0 pid: 80161 (default app) Jul 29 06:12:00.345191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563c3d2fe9c0 pid: 80162 (default app) Jul 29 06:12:00.346188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bb01-919a-49d1-881f-00afc7b51110 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:00.346478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bb01-919a-49d1-881f-00afc7b51110 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:00.346755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bb01-919a-49d1-881f-00afc7b51110 None None] PATH_INFO: `/v3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:00.351515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Fri Jul 29 06:12:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:07.929851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84762f2-adae-47e9-8085-62d3e60471ec None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:07.930401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84762f2-adae-47e9-8085-62d3e60471ec None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:07.930401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84762f2-adae-47e9-8085-62d3e60471ec None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:07.934783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:07.939974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:07.940397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:07.940558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:08.011722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: INFO dbcounter [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] Registered counter for database keystone Jul 29 06:12:08.032844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG oslo_db.sqlalchemy.engines [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80161) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 06:12:08.039020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writer thread running {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 06:12:08.220637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG passlib.handlers.bcrypt [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80161) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 06:12:08.221054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG passlib.handlers.bcrypt [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80161) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 06:12:08.275733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:08.277323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:08.401913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 29 06:12:08.537349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84762f2-adae-47e9-8085-62d3e60471ec None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:08.607845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: INFO dbcounter [None req-f84762f2-adae-47e9-8085-62d3e60471ec None None] Registered counter for database keystone Jul 29 06:12:08.626964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG oslo_db.sqlalchemy.engines [None req-f84762f2-adae-47e9-8085-62d3e60471ec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80162) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 06:12:08.634234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writer thread running {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 06:12:08.758682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84762f2-adae-47e9-8085-62d3e60471ec None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:08.763086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c66e2-8f95-473f-a65a-f13c40d7d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:08.763742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c66e2-8f95-473f-a65a-f13c40d7d1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:08.764027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c66e2-8f95-473f-a65a-f13c40d7d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:08.764306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c66e2-8f95-473f-a65a-f13c40d7d1e9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:08.772841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c66e2-8f95-473f-a65a-f13c40d7d1e9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:08.825695 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb2c66e2-8f95-473f-a65a-f13c40d7d1e9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 06:12:08.825976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c66e2-8f95-473f-a65a-f13c40d7d1e9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:08.836269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Fri Jul 29 06:12:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 06:12:08.842170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:08.856366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2fac4-6548-422f-837b-ab2594cacf76 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:08.860845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f1ed1-083d-4957-b534-1287d6046a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:08.861952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1ed1-083d-4957-b534-1287d6046a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:08.862242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1ed1-083d-4957-b534-1287d6046a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:08.862521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1ed1-083d-4957-b534-1287d6046a94 admin admin] PATH_INFO: `/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:08.864004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f1ed1-083d-4957-b534-1287d6046a94 admin admin] RBAC: Authorizing `identity:get_project(project_id=cc5535113d8e4690869b5d23e1b0a8b2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:08.915016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c6f1ed1-083d-4957-b534-1287d6046a94 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 06:12:08.915357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f1ed1-083d-4957-b534-1287d6046a94 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:08.919943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Fri Jul 29 06:12:08 2022] GET /identity/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:10.904753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d9010-b4c4-43fc-b0ac-a465b97b44d3 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:10.905277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d9010-b4c4-43fc-b0ac-a465b97b44d3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:10.905277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d9010-b4c4-43fc-b0ac-a465b97b44d3 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:10.909388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:10.914628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c67bf2-6e9d-4dbf-9756-20f1d2b391c6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:10.915033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c67bf2-6e9d-4dbf-9756-20f1d2b391c6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:10.915190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c67bf2-6e9d-4dbf-9756-20f1d2b391c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:10.935387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f3c67bf2-6e9d-4dbf-9756-20f1d2b391c6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:10.936934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f3c67bf2-6e9d-4dbf-9756-20f1d2b391c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:10.945429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 29 06:12:10.967396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d9010-b4c4-43fc-b0ac-a465b97b44d3 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:10.979782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d9010-b4c4-43fc-b0ac-a465b97b44d3 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:10.984092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007e26-d418-44bb-8e8a-70487f6549bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:10.984753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007e26-d418-44bb-8e8a-70487f6549bd admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:10.985046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007e26-d418-44bb-8e8a-70487f6549bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:10.985326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007e26-d418-44bb-8e8a-70487f6549bd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:10.993628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007e26-d418-44bb-8e8a-70487f6549bd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:10.995403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007e26-d418-44bb-8e8a-70487f6549bd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:11.004137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Fri Jul 29 06:12:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:12:11.009942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c67bf2-6e9d-4dbf-9756-20f1d2b391c6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:11.018870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c67bf2-6e9d-4dbf-9756-20f1d2b391c6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:11.023424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141136ac-39e7-440a-89f7-2e4d843def7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:11.024045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141136ac-39e7-440a-89f7-2e4d843def7b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:11.024329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141136ac-39e7-440a-89f7-2e4d843def7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:11.024641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141136ac-39e7-440a-89f7-2e4d843def7b admin admin] PATH_INFO: `/v3/users/97726e6afd31490a96b8f997adddff27` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:11.026101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141136ac-39e7-440a-89f7-2e4d843def7b admin admin] RBAC: Authorizing `identity:get_user(user_id=97726e6afd31490a96b8f997adddff27)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:11.027949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141136ac-39e7-440a-89f7-2e4d843def7b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:11.029563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Fri Jul 29 06:12:11 2022] GET /identity/v3/users/97726e6afd31490a96b8f997adddff27 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:13.080596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f197c-8642-42f1-831c-2d0699d63e8f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.081080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f197c-8642-42f1-831c-2d0699d63e8f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.081080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f197c-8642-42f1-831c-2d0699d63e8f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.082449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:13.088440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9bcd-2dff-4d91-87cd-f97e37753b02 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.088867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9bcd-2dff-4d91-87cd-f97e37753b02 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.089168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9bcd-2dff-4d91-87cd-f97e37753b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.106643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5d0b-5cc9-4464-ab30-a47e2366134f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.106936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5d0b-5cc9-4464-ab30-a47e2366134f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.107222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5d0b-5cc9-4464-ab30-a47e2366134f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.108227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bccd9bcd-2dff-4d91-87cd-f97e37753b02 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:13.108575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:13.109560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bccd9bcd-2dff-4d91-87cd-f97e37753b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:13.113393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.113683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.113969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.156747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG passlib.handlers.bcrypt [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80162) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 06:12:13.157172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG passlib.handlers.bcrypt [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80162) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 06:12:13.188977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:13.203233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:13.204869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:13.211648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9bcd-2dff-4d91-87cd-f97e37753b02 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.235391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:13.306880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:13.319111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9bcd-2dff-4d91-87cd-f97e37753b02 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:13.328814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.353240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:13.353892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.354181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.354467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.366203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:13.375081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:13.376145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:13.377166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:13.382570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.430229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1f66c-decc-4261-80f6-f8f827aa14dd None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:13.432514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:13.465616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:13.469012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:13.469881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.470185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.470435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.487194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80b8c-8e55-4b9c-a919-9dda740cca00 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:13.523708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:13.524316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.524628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.524913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.525615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:13.525921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:13.527308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:13.539517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:13.542732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:13.544258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:13.545393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.641460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 06:12:13.646492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf74dc-72d4-4243-b0a8-813286bb567a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:13.647807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.650821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:13.755003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff02e0-b59b-404c-84b9-feafd28d3248 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:13.790451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:13.791302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.791583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.791859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.793855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:13.794499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.794785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.795067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] PATH_INFO: `/v3/users/97726e6afd31490a96b8f997adddff27` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.796238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] RBAC: Authorizing `identity:get_user(user_id=97726e6afd31490a96b8f997adddff27)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:13.797710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:13.799293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/users/97726e6afd31490a96b8f997adddff27 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:13.804804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.809934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:13.811402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:13.819871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:12:13.829846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:13.830943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:13.831732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:12:13.831732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:12:13.831732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:12:13.831732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:13.831732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:13.837834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.893129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:13.907940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474a371-68c6-4f79-8f13-10065973afcd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:13.936844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6029c82-ff03-4fce-879b-8338b8a33926 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:13.941570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:13.942220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.942509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.942789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.944938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:13.946388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:13.948153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:13.953604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:13.970591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937da385-7b21-42ab-a339-b81430210414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:13.971213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937da385-7b21-42ab-a339-b81430210414 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:13.971504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937da385-7b21-42ab-a339-b81430210414 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:13.971785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937da385-7b21-42ab-a339-b81430210414 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:13.972474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937da385-7b21-42ab-a339-b81430210414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:13.972811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937da385-7b21-42ab-a339-b81430210414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:13.974193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937da385-7b21-42ab-a339-b81430210414 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:13.991836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:14.038972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:14.062040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058dd5f-462d-4210-a038-66c3a1909cf3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:14.099400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e603b-f650-4516-84af-0898c6afecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:14.100087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e603b-f650-4516-84af-0898c6afecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:14.100694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e603b-f650-4516-84af-0898c6afecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:14.100694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e603b-f650-4516-84af-0898c6afecf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:14.101434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e603b-f650-4516-84af-0898c6afecf4 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.domain.id=default, user.id=97726e6afd31490a96b8f997adddff27) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:14.101743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e603b-f650-4516-84af-0898c6afecf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:14.103200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e603b-f650-4516-84af-0898c6afecf4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:14.110898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Fri Jul 29 06:12:13 2022] GET /identity/v3/role_assignments?user.id=97726e6afd31490a96b8f997adddff27&scope.domain.id=default&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:16.409425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee9aee-192d-4632-9003-58b83aea8c86 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.409934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee9aee-192d-4632-9003-58b83aea8c86 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.410029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee9aee-192d-4632-9003-58b83aea8c86 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.411575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:16.416505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535377-1486-4c28-92ff-30d177928f63 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.416938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535377-1486-4c28-92ff-30d177928f63 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.417124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535377-1486-4c28-92ff-30d177928f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.442592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8e535377-1486-4c28-92ff-30d177928f63 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:16.443958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8e535377-1486-4c28-92ff-30d177928f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:16.512056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289246-2396-4c6c-b48e-672e625df413 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.512361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289246-2396-4c6c-b48e-672e625df413 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.512652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289246-2396-4c6c-b48e-672e625df413 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.514003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 12/23] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:16.525150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf99c52-37a1-4660-9390-210bf997fd9b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.525437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf99c52-37a1-4660-9390-210bf997fd9b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.525714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf99c52-37a1-4660-9390-210bf997fd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.539441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 12/24] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:16.548018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8fd7a-5310-4487-88cf-010dce946ec4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.548508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8fd7a-5310-4487-88cf-010dce946ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.549011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8fd7a-5310-4487-88cf-010dce946ec4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.550448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 13/25] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:16.555485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a95bda-6a1f-42d6-a534-8cd21bdf248e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.555928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a95bda-6a1f-42d6-a534-8cd21bdf248e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.556341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a95bda-6a1f-42d6-a534-8cd21bdf248e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.561542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-baf99c52-37a1-4660-9390-210bf997fd9b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:16.562742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-baf99c52-37a1-4660-9390-210bf997fd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:16.579141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b9a95bda-6a1f-42d6-a534-8cd21bdf248e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:16.580899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b9a95bda-6a1f-42d6-a534-8cd21bdf248e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:16.685352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 13/26] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:16.686127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf99c52-37a1-4660-9390-210bf997fd9b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:16.711712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 14/27] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:16.736885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:16.738593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a95bda-6a1f-42d6-a534-8cd21bdf248e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:16.748035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:16.812594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf99c52-37a1-4660-9390-210bf997fd9b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:16.852619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:16.853282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.853565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.853841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.945225 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a95bda-6a1f-42d6-a534-8cd21bdf248e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:16.992871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9232a0-2e6a-490b-ac48-648a54d7b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:16.993879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9232a0-2e6a-490b-ac48-648a54d7b29e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:16.994364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9232a0-2e6a-490b-ac48-648a54d7b29e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:16.994674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:16.995070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9232a0-2e6a-490b-ac48-648a54d7b29e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:16.996372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.006331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9232a0-2e6a-490b-ac48-648a54d7b29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-da9232a0-2e6a-490b-ac48-648a54d7b29e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:17.011635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:17.012447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:17.013220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:12:17.013790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:17.014487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:17.015278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:12:17.015278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:12:17.015278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:12:17.015278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:17.015278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 14/28] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:17.016372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:17.017818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 15/29] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:17.019679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11b95-4900-4cb6-bc85-86b6823075de None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.020160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11b95-4900-4cb6-bc85-86b6823075de None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.020649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11b95-4900-4cb6-bc85-86b6823075de None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.022895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 16/30] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:17.027267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11b95-4900-4cb6-bc85-86b6823075de None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:17.129260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:17.138409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:17.143929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d6c94-7b3d-462f-bb4e-ff37044e143b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:17.172803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11b95-4900-4cb6-bc85-86b6823075de None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:17.182788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcea87c-b39a-4a80-bc3b-c9f0093845d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:17.183422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea87c-b39a-4a80-bc3b-c9f0093845d4 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.183705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea87c-b39a-4a80-bc3b-c9f0093845d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.183982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea87c-b39a-4a80-bc3b-c9f0093845d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.184779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcea87c-b39a-4a80-bc3b-c9f0093845d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:17.186311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcea87c-b39a-4a80-bc3b-c9f0093845d4 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.213626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:17.214573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.215071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.215562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.216483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:17.217309 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:17.218881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.228409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 17/31] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:17.229537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:17.328504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 15/32] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 29 06:12:17 2022] POST /identity/v3/roles => generated 219 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:17.330305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd88d1-453a-4199-aede-6b870bf431ad None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.330607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd88d1-453a-4199-aede-6b870bf431ad None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.330890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd88d1-453a-4199-aede-6b870bf431ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.345452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b63a0-4b1a-409f-b938-e5c211db952e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:17.352849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5bbd88d1-453a-4199-aede-6b870bf431ad None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:17.354740 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5bbd88d1-453a-4199-aede-6b870bf431ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:17.375714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:17.386444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:17.387524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.388057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.388560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.389539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:17.390049 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:17.391633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.414409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 18/33] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:17.415646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:17.447149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 16/34] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:17.448214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd88d1-453a-4199-aede-6b870bf431ad None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:17.575342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee9c2-cd56-4e8f-8dc6-b1b82120ab23 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:17.595611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:17.611536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd88d1-453a-4199-aede-6b870bf431ad None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:17.619206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:17.620149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.620794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.621398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] PATH_INFO: `/v3/users/97726e6afd31490a96b8f997adddff27` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.622885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] RBAC: Authorizing `identity:get_user(user_id=97726e6afd31490a96b8f997adddff27)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:17.624659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.626485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 19/35] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:17 2022] GET /identity/v3/users/97726e6afd31490a96b8f997adddff27 => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:17.627419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:17.652904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:17.653563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.654300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.654300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.655200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:17.657839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.692432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:17.695745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 17/36] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:12:17 2022] POST /identity/v3/projects => generated 301 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:17.697283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:17.753794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb082b3-5850-462b-95f8-a23e27aba490 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:17.800068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:17.800756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.801049 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.801336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.803714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:17.804459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.850052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1c5e-e9a8-4923-8670-9eeea50461c3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:17.893887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fb3d9-eb83-4b5c-b017-580939e6e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:17.895098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb3d9-eb83-4b5c-b017-580939e6e1fc None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.895613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb3d9-eb83-4b5c-b017-580939e6e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.896103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb3d9-eb83-4b5c-b017-580939e6e1fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.898556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fb3d9-eb83-4b5c-b017-580939e6e1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:17.900373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fb3d9-eb83-4b5c-b017-580939e6e1fc None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:17.902466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 18/37] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:17.903859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a890c-b581-48c9-a897-70c4a0419fcf None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.904344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a890c-b581-48c9-a897-70c4a0419fcf None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.904858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a890c-b581-48c9-a897-70c4a0419fcf None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.906267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 19/38] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:17.912318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce455d6-dd64-4522-9344-12513ff4bee7 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:17.913655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce455d6-dd64-4522-9344-12513ff4bee7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:17.914125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce455d6-dd64-4522-9344-12513ff4bee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:17.934013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-fce455d6-dd64-4522-9344-12513ff4bee7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:17.935475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fce455d6-dd64-4522-9344-12513ff4bee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:17.937659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:17.940857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 20/39] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 29 06:12:17 2022] POST /identity/v3/roles => generated 225 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:17.941847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:18.032010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 20/40] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:18.058276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce455d6-dd64-4522-9344-12513ff4bee7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:18.063350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf92d9-b8d0-46eb-86c5-98b97933f13a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:18.079948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:18.108379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada27c7-1032-44af-938f-9df2597eec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:18.109041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada27c7-1032-44af-938f-9df2597eec8b None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:18.109333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada27c7-1032-44af-938f-9df2597eec8b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:18.109624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada27c7-1032-44af-938f-9df2597eec8b None admin] PATH_INFO: `/v3/domains/default/users/97726e6afd31490a96b8f997adddff27/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:18.111969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada27c7-1032-44af-938f-9df2597eec8b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=97726e6afd31490a96b8f997adddff27, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:18.114032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada27c7-1032-44af-938f-9df2597eec8b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:18.136283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 21/41] 10.222.0.33 () {58 vars in 1361 bytes} [Fri Jul 29 06:12:17 2022] PUT /identity/v3/domains/default/users/97726e6afd31490a96b8f997adddff27/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:18.150780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0d87-a196-4281-8367-b2b19b959556 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:18.151223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0d87-a196-4281-8367-b2b19b959556 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:18.151662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0d87-a196-4281-8367-b2b19b959556 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:18.155400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 22/42] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:18.161023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fe04d-f857-46ef-98f7-19b5581d407e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:18.161341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fe04d-f857-46ef-98f7-19b5581d407e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:18.161605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fe04d-f857-46ef-98f7-19b5581d407e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:18.181191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e64fe04d-f857-46ef-98f7-19b5581d407e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:18.182792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e64fe04d-f857-46ef-98f7-19b5581d407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:18.195098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce455d6-dd64-4522-9344-12513ff4bee7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:18.245072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:18.246089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:18.246583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:18.246949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:18.247770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:18.249482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:18.262455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:18.276793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 21/43] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 29 06:12:18 2022] POST /identity/v3/roles => generated 223 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:18.289993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 23/44] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:18.313387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:18.430044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96026a-edbb-4108-90d6-8eaf828cc915 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:18.467620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d44924-1b10-4c87-a732-e5170c67b85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:18.468253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44924-1b10-4c87-a732-e5170c67b85f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:18.468557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44924-1b10-4c87-a732-e5170c67b85f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:18.468846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44924-1b10-4c87-a732-e5170c67b85f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:18.470038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44924-1b10-4c87-a732-e5170c67b85f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:18.471485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44924-1b10-4c87-a732-e5170c67b85f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:18.473665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 22/45] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:18.479970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fe04d-f857-46ef-98f7-19b5581d407e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:18.578918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fe04d-f857-46ef-98f7-19b5581d407e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:18.581480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:18.613958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0ec6f-8e46-46a4-a508-0bb58c9f9491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:18.614587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0ec6f-8e46-46a4-a508-0bb58c9f9491 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:18.614872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0ec6f-8e46-46a4-a508-0bb58c9f9491 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:18.615153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0ec6f-8e46-46a4-a508-0bb58c9f9491 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:18.615939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0ec6f-8e46-46a4-a508-0bb58c9f9491 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:18.617456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0ec6f-8e46-46a4-a508-0bb58c9f9491 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:18.644574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:12:18 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:20.185039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1f250-ea55-4ec3-9841-bd90b5194f02 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.185553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1f250-ea55-4ec3-9841-bd90b5194f02 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.185634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1f250-ea55-4ec3-9841-bd90b5194f02 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.186968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:20.192149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebddbc-2018-4da2-a90b-96c951a52a88 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.192623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebddbc-2018-4da2-a90b-96c951a52a88 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.192769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebddbc-2018-4da2-a90b-96c951a52a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.211475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6bebddbc-2018-4da2-a90b-96c951a52a88 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:20.212697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6bebddbc-2018-4da2-a90b-96c951a52a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:20.278695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 25/48] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:20.301522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1f250-ea55-4ec3-9841-bd90b5194f02 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:20.367861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:20.403872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1f250-ea55-4ec3-9841-bd90b5194f02 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:20.438848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:20.439408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.439693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.439968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.451363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:20.458038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:20.458927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:20.459728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 24/49] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:20.465493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebddbc-2018-4da2-a90b-96c951a52a88 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:20.549030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:20.565284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebddbc-2018-4da2-a90b-96c951a52a88 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:20.600705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e9004-7794-4981-96b1-3ae306d9aa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:20.601542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9004-7794-4981-96b1-3ae306d9aa6d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.601829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9004-7794-4981-96b1-3ae306d9aa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.602110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9004-7794-4981-96b1-3ae306d9aa6d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.602830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e9004-7794-4981-96b1-3ae306d9aa6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:20.603145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e9004-7794-4981-96b1-3ae306d9aa6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:20.604584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e9004-7794-4981-96b1-3ae306d9aa6d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:20.612621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 26/50] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:20.618753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:20.638317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf88476-9c1d-4889-b759-cf249b2cea27 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.638612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf88476-9c1d-4889-b759-cf249b2cea27 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.638899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf88476-9c1d-4889-b759-cf249b2cea27 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.640252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 27/51] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:20.645291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ba7a3-2ff1-44ac-9e11-63cac5327354 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.645589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ba7a3-2ff1-44ac-9e11-63cac5327354 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.645878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ba7a3-2ff1-44ac-9e11-63cac5327354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.664912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fd6ba7a3-2ff1-44ac-9e11-63cac5327354 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:20.666144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fd6ba7a3-2ff1-44ac-9e11-63cac5327354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:20.724728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b126ef-3cd2-4af5-81c9-6bfe9c7d9186 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:20.734147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 28/52] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:20.753872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:20.762600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:20.763250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.763625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.763918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] PATH_INFO: `/v3/users/97726e6afd31490a96b8f997adddff27` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.765375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] RBAC: Authorizing `identity:get_user(user_id=97726e6afd31490a96b8f997adddff27)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:20.766834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:20.767593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ba7a3-2ff1-44ac-9e11-63cac5327354 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:20.768445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 25/53] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:20 2022] GET /identity/v3/users/97726e6afd31490a96b8f997adddff27 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:20.774246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:20.846005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:20.871095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ba7a3-2ff1-44ac-9e11-63cac5327354 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:20.875905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6ee23-92a1-46f6-b132-7e5731ca13e8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:20.907981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:20.908658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.908949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.909234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.910286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:20.910400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:20.910938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:20.911228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:20.911515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:20.911810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:20.913469 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:20.914009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 29/54] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:20.914933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:20.916622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 26/55] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:20.919531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:20.922084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:20.982663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:21.017420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa5e6d-21ba-44c1-901c-b92d90470a44 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:21.023166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7e5cf-0365-432b-add8-4441b1d40960 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:21.056174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa9f8e-fc95-46f2-b757-f6aa6d6a0e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:21.056634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327da61-17c8-4972-9d2e-672f7d049dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:21.056842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa9f8e-fc95-46f2-b757-f6aa6d6a0e9f None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:21.057135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa9f8e-fc95-46f2-b757-f6aa6d6a0e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:21.057265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327da61-17c8-4972-9d2e-672f7d049dee None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:21.057424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa9f8e-fc95-46f2-b757-f6aa6d6a0e9f None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:21.057562 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327da61-17c8-4972-9d2e-672f7d049dee None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:21.057847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327da61-17c8-4972-9d2e-672f7d049dee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:21.058159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa9f8e-fc95-46f2-b757-f6aa6d6a0e9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:21.058634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327da61-17c8-4972-9d2e-672f7d049dee None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.domain.id=default, user.id=97726e6afd31490a96b8f997adddff27) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:21.058951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327da61-17c8-4972-9d2e-672f7d049dee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:21.059618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa9f8e-fc95-46f2-b757-f6aa6d6a0e9f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:21.060398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327da61-17c8-4972-9d2e-672f7d049dee None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:21.069644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 27/56] 10.222.0.33 () {56 vars in 1322 bytes} [Fri Jul 29 06:12:20 2022] GET /identity/v3/role_assignments?user.id=97726e6afd31490a96b8f997adddff27&scope.domain.id=default&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:21.081127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:21.090170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 30/57] 10.222.0.33 () {60 vars in 1140 bytes} [Fri Jul 29 06:12:20 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:12:23.129934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475d8c6-932b-493b-aa60-15220500a23f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:23.130438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475d8c6-932b-493b-aa60-15220500a23f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:23.130539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475d8c6-932b-493b-aa60-15220500a23f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:23.132505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 28/58] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:23.137572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aee57-835f-4781-a328-f599b45e4f2e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:23.137950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aee57-835f-4781-a328-f599b45e4f2e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:23.138145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aee57-835f-4781-a328-f599b45e4f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:23.155815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-556aee57-835f-4781-a328-f599b45e4f2e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:23.157028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-556aee57-835f-4781-a328-f599b45e4f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:23.221783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 31/59] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:23.245463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475d8c6-932b-493b-aa60-15220500a23f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:23.347638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475d8c6-932b-493b-aa60-15220500a23f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:23.373220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:23.382144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2956b-8dfd-429e-bbe3-adfa377a0d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:23.382721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2956b-8dfd-429e-bbe3-adfa377a0d54 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:23.383002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2956b-8dfd-429e-bbe3-adfa377a0d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:23.383278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2956b-8dfd-429e-bbe3-adfa377a0d54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:23.384417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2956b-8dfd-429e-bbe3-adfa377a0d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:23.385875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2956b-8dfd-429e-bbe3-adfa377a0d54 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:23.387448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 29/60] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:23.393021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aee57-835f-4781-a328-f599b45e4f2e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:23.492855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aee57-835f-4781-a328-f599b45e4f2e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:23.518249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:23.526327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55d242-4e4c-40b3-a2cd-389e59537c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:23.526953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55d242-4e4c-40b3-a2cd-389e59537c22 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:23.527240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55d242-4e4c-40b3-a2cd-389e59537c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:23.527521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55d242-4e4c-40b3-a2cd-389e59537c22 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:23.528223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55d242-4e4c-40b3-a2cd-389e59537c22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:23.529811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55d242-4e4c-40b3-a2cd-389e59537c22 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:23.555938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 32/61] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:12:23 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:26.716332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3166-7f75-4a8a-b1fb-e021e4b48478 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:26.716909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3166-7f75-4a8a-b1fb-e021e4b48478 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:26.716909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3166-7f75-4a8a-b1fb-e021e4b48478 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:26.720327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 30/62] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:26.731723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ca90-6a26-4660-9971-127d87acc776 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:26.732193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ca90-6a26-4660-9971-127d87acc776 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:26.732289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ca90-6a26-4660-9971-127d87acc776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:26.766821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e435ca90-6a26-4660-9971-127d87acc776 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:26.768043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e435ca90-6a26-4660-9971-127d87acc776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:26.874790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 33/63] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:26.894272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bff4b4-87f3-4a56-847c-1744452125ed None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:26.894574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bff4b4-87f3-4a56-847c-1744452125ed None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:26.894852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bff4b4-87f3-4a56-847c-1744452125ed None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:26.896441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 31/64] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:26.902091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803d8f-e26e-4922-b4e1-ad27dd82bbf6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:26.902393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803d8f-e26e-4922-b4e1-ad27dd82bbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:26.902682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803d8f-e26e-4922-b4e1-ad27dd82bbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:26.910558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bff4b4-87f3-4a56-847c-1744452125ed None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:26.927743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3a803d8f-e26e-4922-b4e1-ad27dd82bbf6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:26.929533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3a803d8f-e26e-4922-b4e1-ad27dd82bbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:27.052994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 34/65] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:27.055478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dbf69-a9d0-402f-8c81-8c688b1c5be8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.055973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dbf69-a9d0-402f-8c81-8c688b1c5be8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.056446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dbf69-a9d0-402f-8c81-8c688b1c5be8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.062567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 35/66] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:27.065365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a9bbf-190e-416a-85e2-a5787d802abb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.065660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a9bbf-190e-416a-85e2-a5787d802abb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.065946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a9bbf-190e-416a-85e2-a5787d802abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.081009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bff4b4-87f3-4a56-847c-1744452125ed None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.092659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-300a9bbf-190e-416a-85e2-a5787d802abb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:27.094843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-300a9bbf-190e-416a-85e2-a5787d802abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:27.121269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.121936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.122229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.122511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.134902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:27.139863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:27.140711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:27.141554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 32/67] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:27.142193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.143590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.176361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 36/68] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:27.177580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b6f14-2149-4ddc-a66e-6df7f8414a54 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.177911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b6f14-2149-4ddc-a66e-6df7f8414a54 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.178791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b6f14-2149-4ddc-a66e-6df7f8414a54 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.184150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 37/69] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:27.184150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b6f14-2149-4ddc-a66e-6df7f8414a54 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.213662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.267750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d76f4d-aba8-467a-8842-6b08703cf94c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.296482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b6f14-2149-4ddc-a66e-6df7f8414a54 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.305403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7313791-0142-4103-be1e-aaec7d77b39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.306276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313791-0142-4103-be1e-aaec7d77b39f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.306561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313791-0142-4103-be1e-aaec7d77b39f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.306840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313791-0142-4103-be1e-aaec7d77b39f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.317634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7313791-0142-4103-be1e-aaec7d77b39f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c7313791-0142-4103-be1e-aaec7d77b39f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:27.323303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:27.324133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:27.325354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 33/70] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:27.326133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7da1b-a6d0-4121-81fd-568c06bc0e59 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.326366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7da1b-a6d0-4121-81fd-568c06bc0e59 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.326831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7da1b-a6d0-4121-81fd-568c06bc0e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.341245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.341888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.342174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.342453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.343181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:27.343489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:27.344926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:27.346054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-53c7da1b-a6d0-4121-81fd-568c06bc0e59 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:27.348176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-53c7da1b-a6d0-4121-81fd-568c06bc0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:27.356345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 38/71] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:27.357225 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.394598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.420857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 34/72] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:27.421506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7da1b-a6d0-4121-81fd-568c06bc0e59 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.467125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.473010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4566edc-ef27-48b7-8686-08c8f44b0a3a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.506295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100dd7e0-7407-4533-9c13-45164788cabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.506945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100dd7e0-7407-4533-9c13-45164788cabf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.507226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100dd7e0-7407-4533-9c13-45164788cabf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.507504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100dd7e0-7407-4533-9c13-45164788cabf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.517563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100dd7e0-7407-4533-9c13-45164788cabf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:27.520984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7da1b-a6d0-4121-81fd-568c06bc0e59 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-100dd7e0-7407-4533-9c13-45164788cabf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:27.523531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:27.524380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:27.525352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 39/73] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:27.526217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964165f5-6288-4f93-903b-4443f203d621 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.526217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964165f5-6288-4f93-903b-4443f203d621 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.526501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964165f5-6288-4f93-903b-4443f203d621 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.527754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 40/74] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:27.528455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964165f5-6288-4f93-903b-4443f203d621 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.559362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.559922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.560213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.560496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.561264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:27.561578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:27.562979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:27.575039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 35/75] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:27.575039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.628074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.635750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964165f5-6288-4f93-903b-4443f203d621 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.668636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.669283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.669571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.669852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.679144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a3f09-ed99-4181-b8a5-bd782951ea15 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.684179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:27.685704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:27.686480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.687368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 41/76] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:27.688281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.720514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6804c82-3e11-4b3b-9965-28f6b261363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.721355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804c82-3e11-4b3b-9965-28f6b261363f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.721646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804c82-3e11-4b3b-9965-28f6b261363f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.721931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804c82-3e11-4b3b-9965-28f6b261363f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.723134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6804c82-3e11-4b3b-9965-28f6b261363f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:27.724621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6804c82-3e11-4b3b-9965-28f6b261363f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:27.726288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 36/77] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:27.729496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3cee-bcac-41fb-89b5-1c69f5918896 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.729496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3cee-bcac-41fb-89b5-1c69f5918896 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.729496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3cee-bcac-41fb-89b5-1c69f5918896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.746258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9a6a3cee-bcac-41fb-89b5-1c69f5918896 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:27.747454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9a6a3cee-bcac-41fb-89b5-1c69f5918896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:27.791186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252320b4-5db9-492e-ab5d-3fbccbd4685c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.812572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 37/78] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:27.813326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3cee-bcac-41fb-89b5-1c69f5918896 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.828990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.830903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.831695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.832409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.833610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:27.833926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:27.836166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:27.844825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 42/79] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:27.850667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.869578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.916458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3cee-bcac-41fb-89b5-1c69f5918896 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.918800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:27.950728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.951383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.951669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.951948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] PATH_INFO: `/v3/users/97726e6afd31490a96b8f997adddff27` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:27.953136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] RBAC: Authorizing `identity:get_user(user_id=97726e6afd31490a96b8f997adddff27)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:27.953706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de94174-051e-46ad-9326-6fe1405d76d2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:27.954579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:27.956457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 38/80] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/users/97726e6afd31490a96b8f997adddff27 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:27.957215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:27.988435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:27.989123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:27.989411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:27.989691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.055430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8355e-2387-44b8-a340-c415040048d0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.057959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.059434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:28.076656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:28.094486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.095202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.095495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.095778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.096468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.097958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:28.125544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 39/81] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:12:27 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:28.126600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.170320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 43/82] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:28.170320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.181622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:28.262324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbacb07-db4f-4183-b900-f44804464e72 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.292113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d138347-8718-4169-92a1-9f655fe59480 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.310812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.311481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.311768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.312574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.312675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:28.325572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:28.330978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:28.331844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:28.332672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 40/83] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:28.333303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.337256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.337922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.338206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.338477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.339751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.341270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:28.342907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 44/84] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:28.343663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.447823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:28.465981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4da344-eff3-4169-b5dc-ccacf0065243 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.480755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1afd3d-0f09-4ae5-ac38-7bf7a86c4e15 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.515395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.516366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.516683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.517007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.519834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.520523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.520919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.521208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.523054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.524527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:28.526267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 45/85] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:28.527251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.594437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:28.611702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.614676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:28.649673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fb103-db6e-4ed1-9794-5812aea15e8f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.695246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.695877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.696169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.696455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.697211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:28.697527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.698927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:28.712504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 46/86] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:28.713463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.742759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:28.767393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 41/87] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:28.768575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.836439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7af08f-c0a7-46c0-9f57-c8eb71fb98dd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.852897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:28.883235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.883881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.884176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.884467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.898522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af4f4-982f-4169-b2a4-0a255f46b22f None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:28.936204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:28.936889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:28.937183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:28.937470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:28.938267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=1829d1e77c554b12a2c6482e57c9fe41) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:28.938586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.940046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:28.947186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 42/88] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/role_assignments?user.id=1829d1e77c554b12a2c6482e57c9fe41&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:28.948180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:28.981415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:28.983694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:29.021260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:29.136198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d48bfe-2c39-4048-a06a-547e93c25af6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:29.159961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 47/89] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:29.161748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:29.189630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:29.204129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:29.204824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:29.205115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:29.205657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:29.207441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:29.209535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:29.211191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 43/90] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:28 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:29.213267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:29.343900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca915b-9c9b-4b2a-920e-781a730d9d74 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:29.418900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d620c46-d9c1-4d4f-95e9-90d4dfa45ba9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:29.423143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:29.423816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:29.424109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:29.424395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:29.432403 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:29.433921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:29.435675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 48/91] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:29 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:29.436450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:29.467228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:29.497410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:29.498078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:29.498370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:29.498656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:29.504372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:29.505920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:29.509512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 44/92] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:29 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:29.510442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:29.576906 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:29.635920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517f7e7-b3c2-4fc5-923d-086a751c6e65 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:29.659534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d47f82-3896-4147-b959-f3b262aeeb76 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:29.713664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:29.714694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:29.714989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:29.715275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:29.719866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:29.719866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:29.720073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:29.720568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:29.721146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=97726e6afd31490a96b8f997adddff27) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:29.721468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:29.722922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:29.743238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 45/93] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:29 2022] GET /identity/v3/role_assignments?user.id=97726e6afd31490a96b8f997adddff27&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:29.744195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:29.840640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:29.848571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:29.872941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:29.948490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:29.970215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 49/94] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:29.977805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:29.978720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1bc59-7061-4ce7-8fb6-5859b98ea614 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:30.062837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:30.063500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:30.063797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:30.064086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:30.065077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] RBAC: Adding query filter params (role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=1829d1e77c554b12a2c6482e57c9fe41) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:30.065404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:30.067102 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:30.084476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 46/95] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:29 2022] GET /identity/v3/role_assignments?user.id=1829d1e77c554b12a2c6482e57c9fe41&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:30.125696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a781c-e3ed-4c28-8b3c-868000e50859 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:30.180059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365fd8d-c41f-4d69-8006-f1704c424018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:30.180813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365fd8d-c41f-4d69-8006-f1704c424018 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:30.181101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365fd8d-c41f-4d69-8006-f1704c424018 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:30.181385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365fd8d-c41f-4d69-8006-f1704c424018 None admin] PATH_INFO: `/v3/projects/d02a5a36a45a44e8b905905893ed0361` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:30.185262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365fd8d-c41f-4d69-8006-f1704c424018 None admin] RBAC: Authorizing `identity:get_project(project_id=d02a5a36a45a44e8b905905893ed0361)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:30.186777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365fd8d-c41f-4d69-8006-f1704c424018 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:30.188914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 50/96] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:29 2022] GET /identity/v3/projects/d02a5a36a45a44e8b905905893ed0361 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:30.195221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:30.342159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d341a-7f28-4f18-98bf-8842e3f4d1f5 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:30.368405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:30.392107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe70510-5502-4f0a-9820-ad750727f5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:30.392837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70510-5502-4f0a-9820-ad750727f5ab None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:30.393119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70510-5502-4f0a-9820-ad750727f5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:30.393396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70510-5502-4f0a-9820-ad750727f5ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:30.394299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70510-5502-4f0a-9820-ad750727f5ab None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=d02a5a36a45a44e8b905905893ed0361, user.id=1829d1e77c554b12a2c6482e57c9fe41) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:30.394615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70510-5502-4f0a-9820-ad750727f5ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:30.396086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70510-5502-4f0a-9820-ad750727f5ab None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:30.404029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 47/97] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:30 2022] GET /identity/v3/role_assignments?user.id=1829d1e77c554b12a2c6482e57c9fe41&scope.project.id=d02a5a36a45a44e8b905905893ed0361&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:30.538616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5d40-47b9-4026-9b9c-7d6eff8cd928 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:30.538923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5d40-47b9-4026-9b9c-7d6eff8cd928 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:30.539180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5d40-47b9-4026-9b9c-7d6eff8cd928 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:30.540823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 51/98] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:30.546312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d5f34-d99a-45b6-81b3-304afaa179c6 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:30.546600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d5f34-d99a-45b6-81b3-304afaa179c6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:30.546886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d5f34-d99a-45b6-81b3-304afaa179c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:30.577885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3a3d5f34-d99a-45b6-81b3-304afaa179c6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:30.579217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3a3d5f34-d99a-45b6-81b3-304afaa179c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:30.671083 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 48/99] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:30.707516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5d40-47b9-4026-9b9c-7d6eff8cd928 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:30.801511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:30.866823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5d40-47b9-4026-9b9c-7d6eff8cd928 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:30.916139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e739b-ae70-44ff-a43f-aa188c5cf0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:30.916835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e739b-ae70-44ff-a43f-aa188c5cf0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:30.917127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e739b-ae70-44ff-a43f-aa188c5cf0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:30.917409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e739b-ae70-44ff-a43f-aa188c5cf0c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:30.936589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e739b-ae70-44ff-a43f-aa188c5cf0c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3e2e739b-ae70-44ff-a43f-aa188c5cf0c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:30.942044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:30.942900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:30.943664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 52/100] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:30.950194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d5f34-d99a-45b6-81b3-304afaa179c6 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:30.987977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805f075-86c3-4e09-9ec6-9d1ce4be6491 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:30.988325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805f075-86c3-4e09-9ec6-9d1ce4be6491 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:30.988643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805f075-86c3-4e09-9ec6-9d1ce4be6491 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:30.991349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 53/101] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:31.003564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a608ba-ee2f-47b1-85f7-2ca5ed1f0655 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:31.003838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a608ba-ee2f-47b1-85f7-2ca5ed1f0655 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:31.004115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a608ba-ee2f-47b1-85f7-2ca5ed1f0655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:31.038260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-32a608ba-ee2f-47b1-85f7-2ca5ed1f0655 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:31.039475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-32a608ba-ee2f-47b1-85f7-2ca5ed1f0655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:31.134641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d5f34-d99a-45b6-81b3-304afaa179c6 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:31.146326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:31.176318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:31.177207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:31.177741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:31.178390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:31.179384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:31.179844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:31.181421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:31.191000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 49/102] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:31.200843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:31.201693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 54/103] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:31.240129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a608ba-ee2f-47b1-85f7-2ca5ed1f0655 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:31.313785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea515c-6b7c-4d26-ab52-681ff10e2722 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:31.343028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:31.353038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:31.353968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:31.354404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:31.354829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] PATH_INFO: `/v3/users/8b67a047eae74976b581ea18ee84c963` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:31.372284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] RBAC: Authorizing `identity:get_user(user_id=8b67a047eae74976b581ea18ee84c963)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:31.374053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:31.376003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 50/104] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:31 2022] GET /identity/v3/users/8b67a047eae74976b581ea18ee84c963 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:31.385667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:31.444159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:31.484457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a608ba-ee2f-47b1-85f7-2ca5ed1f0655 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:31.547301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:31.547963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:31.548253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:31.548555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:31.550663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:31.552151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:31.553861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 55/105] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:31.561109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:31.578908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75d7e9-13d0-42a6-bdcd-c4c7d4b3e349 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:31.656230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:31.656903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:31.657195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:31.657479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:31.779198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:31.780705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:31.794784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e880-285d-441d-84f2-ccac546d6c24 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:31.814603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:31.833677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:31.888479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:31.889173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:31.889429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:31.889714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:31.890422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:31.891892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:31.959097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 56/106] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:12:31 2022] POST /identity/v3/projects => generated 302 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:31.964303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 51/107] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:31.969872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:32.127914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05d473-e4e5-4236-8593-39fecf10d40a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:32.175955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77521d05-6d41-46fb-ad7a-12da16b62dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:32.176654 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521d05-6d41-46fb-ad7a-12da16b62dda None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:32.176946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521d05-6d41-46fb-ad7a-12da16b62dda None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:32.177233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521d05-6d41-46fb-ad7a-12da16b62dda None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:32.178478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77521d05-6d41-46fb-ad7a-12da16b62dda None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:32.179946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77521d05-6d41-46fb-ad7a-12da16b62dda None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:32.182042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 57/108] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:31 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:32.187623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:32.245654 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e875a56-7909-4c19-ba8c-0ccc58cb91f4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:32.245953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e875a56-7909-4c19-ba8c-0ccc58cb91f4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:32.246471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e875a56-7909-4c19-ba8c-0ccc58cb91f4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:32.248058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 58/109] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:32.253587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a973-4406-471c-bb59-92e4ac730e5a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:32.254260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a973-4406-471c-bb59-92e4ac730e5a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:32.254742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a973-4406-471c-bb59-92e4ac730e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:32.283294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6294a973-4406-471c-bb59-92e4ac730e5a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:32.284930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6294a973-4406-471c-bb59-92e4ac730e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:32.327270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:32.363117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1d621-6d82-43bd-8695-9c4adf37f75e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:32.389565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 59/110] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:32.417390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294a973-4406-471c-bb59-92e4ac730e5a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:32.436887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:32.448806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:32.450121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:32.451245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:32.452494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:32.453536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] RBAC: Adding query filter params (role.id=f051e915d2d1479ba724a09298df84b7, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=8b67a047eae74976b581ea18ee84c963) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:32.454223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:32.457455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:32.468708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 52/111] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:32 2022] GET /identity/v3/role_assignments?user.id=8b67a047eae74976b581ea18ee84c963&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=f051e915d2d1479ba724a09298df84b7 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:32.603069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294a973-4406-471c-bb59-92e4ac730e5a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:32.649711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d85264-1367-4162-b05c-2b32de91c023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:32.650312 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85264-1367-4162-b05c-2b32de91c023 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:32.650604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85264-1367-4162-b05c-2b32de91c023 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:32.650891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85264-1367-4162-b05c-2b32de91c023 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:32.668910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d85264-1367-4162-b05c-2b32de91c023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b8d85264-1367-4162-b05c-2b32de91c023 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:32.674026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:32.674889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:32.676083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 60/112] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:32.683536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:32.848995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b804e-f364-4501-8ea6-3054b66936fb None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:32.849285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b804e-f364-4501-8ea6-3054b66936fb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:32.849774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b804e-f364-4501-8ea6-3054b66936fb None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:32.851502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 61/113] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:32.861917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5cf2a-6383-4ee9-a9b4-6543cfba545e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:32.862405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5cf2a-6383-4ee9-a9b4-6543cfba545e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:32.862837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5cf2a-6383-4ee9-a9b4-6543cfba545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:32.890280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-30b5cf2a-6383-4ee9-a9b4-6543cfba545e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:32.891734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-30b5cf2a-6383-4ee9-a9b4-6543cfba545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:32.907591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b82b-3bf5-4394-83cf-78bee2e5cf02 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:32.919634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:32.999670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bc594-d771-45d5-87dc-797301f5c8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.000335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bc594-d771-45d5-87dc-797301f5c8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.000621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bc594-d771-45d5-87dc-797301f5c8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.000896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bc594-d771-45d5-87dc-797301f5c8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.001629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bc594-d771-45d5-87dc-797301f5c8d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:33.001940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bc594-d771-45d5-87dc-797301f5c8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:33.003649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bc594-d771-45d5-87dc-797301f5c8d2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:33.023286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 53/114] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:33.024379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595c7a-5985-47dd-9503-a76cd78b588d None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.024699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595c7a-5985-47dd-9503-a76cd78b588d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.024984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595c7a-5985-47dd-9503-a76cd78b588d None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.026659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 54/115] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:33.030668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36595c7a-5985-47dd-9503-a76cd78b588d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.033853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 62/116] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:33.035297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894da95-dc0d-4ad0-8dbd-2380805ba523 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.035828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894da95-dc0d-4ad0-8dbd-2380805ba523 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.038911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894da95-dc0d-4ad0-8dbd-2380805ba523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.081781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f894da95-dc0d-4ad0-8dbd-2380805ba523 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:33.083020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f894da95-dc0d-4ad0-8dbd-2380805ba523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:33.172526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:33.193124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 63/117] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:33.194056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894da95-dc0d-4ad0-8dbd-2380805ba523 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.232199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36595c7a-5985-47dd-9503-a76cd78b588d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:33.268568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.269465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.270104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.270566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.272047 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:33.273788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:33.275669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 55/118] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:33 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:33.281052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.372227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:33.426067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894da95-dc0d-4ad0-8dbd-2380805ba523 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:33.428206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ebfac-96ab-4566-9ec1-28b2b035a14f None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:33.471967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.472956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.473485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.473942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.486390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:33.492676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:33.493537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:33.494295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:33.496276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 56/119] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:12:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:33.500685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:33.501710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.514690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.515796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.516278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.520016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.551332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:33.569433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:33.570555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:33.571526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 64/120] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:33.581922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.614409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1decd4-ed78-430f-9082-20b1884446b3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:33.651038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.651955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.652389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.652836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.654149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:33.655789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:33.657724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 57/121] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:33 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:33.658685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.731848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:33.764339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b2b00-9a1f-4eff-89d6-d906e1aaf741 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:33.800106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.801018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.801471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.801895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.803485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:33.804010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:33.804892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f537c06-b325-4aee-a384-73fe805a9936 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:33.805768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:33.820387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 58/122] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:33.822171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.865389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954224f-f35d-462a-bd61-f295193367b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.866406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954224f-f35d-462a-bd61-f295193367b5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.866771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954224f-f35d-462a-bd61-f295193367b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.867175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954224f-f35d-462a-bd61-f295193367b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.867995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954224f-f35d-462a-bd61-f295193367b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:33.868440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954224f-f35d-462a-bd61-f295193367b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:33.870097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954224f-f35d-462a-bd61-f295193367b5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:33.876105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:33.881966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 65/123] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:12:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:33.883345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2f88-ef9b-41ab-b2be-2a0f73b82d22 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.883658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2f88-ef9b-41ab-b2be-2a0f73b82d22 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.883928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2f88-ef9b-41ab-b2be-2a0f73b82d22 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.885715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 66/124] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:33.886482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f2f88-ef9b-41ab-b2be-2a0f73b82d22 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:33.945963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ab3c-d884-42b0-b5c3-82c54904b9c1 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:33.961122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:33.984306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5291be0-931c-4dd7-bedf-2afd09c668cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:33.985074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5291be0-931c-4dd7-bedf-2afd09c668cc None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:33.985427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5291be0-931c-4dd7-bedf-2afd09c668cc None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:33.985814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5291be0-931c-4dd7-bedf-2afd09c668cc None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/1829d1e77c554b12a2c6482e57c9fe41/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:33.988024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5291be0-931c-4dd7-bedf-2afd09c668cc None admin] RBAC: Authorizing `identity:create_grant(project_id=d90586d8e9094e65ad00907c4b6c8ddd, user_id=1829d1e77c554b12a2c6482e57c9fe41, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:33.989664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5291be0-931c-4dd7-bedf-2afd09c668cc None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:34.006289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 59/125] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:33 2022] PUT /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/1829d1e77c554b12a2c6482e57c9fe41/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:34.013811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46a1f-f144-4b72-a9fc-b329ca699f26 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:34.015185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f2f88-ef9b-41ab-b2be-2a0f73b82d22 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:34.017061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46a1f-f144-4b72-a9fc-b329ca699f26 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:34.017379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46a1f-f144-4b72-a9fc-b329ca699f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:34.061112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-92f46a1f-f144-4b72-a9fc-b329ca699f26 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:34.062386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-92f46a1f-f144-4b72-a9fc-b329ca699f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:34.105403 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:34.106211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:34.106505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:34.106795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] PATH_INFO: `/v3/users/97726e6afd31490a96b8f997adddff27` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:34.113488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] RBAC: Authorizing `identity:get_user(user_id=97726e6afd31490a96b8f997adddff27)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:34.115002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:34.117545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 67/126] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:33 2022] GET /identity/v3/users/97726e6afd31490a96b8f997adddff27 => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:34.118455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:34.177576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 60/127] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:34.180071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46a1f-f144-4b72-a9fc-b329ca699f26 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:34.237477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:34.278330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c1d30-1b36-40f7-b659-d6ab3655cd51 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:34.297991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46a1f-f144-4b72-a9fc-b329ca699f26 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:34.301018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:34.314180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:34.314885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:34.315275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:34.315566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:34.316828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:34.318317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:34.319961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 68/128] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:34 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:34.326809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:34.346786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:34.347458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:34.347740 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:34.348172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:34.352218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:34.353986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:34.355714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 61/129] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:34 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:34.356622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:34.439810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9645802-1208-4d01-b51a-33bd4bf37cb4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:34.478508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:34.478508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:34.478508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:34.478508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:34.481383 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:34.495013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:34.503171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:34.504009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:34.504868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:34.512588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 69/130] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:34.512588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:34.604811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251406d-4cef-4f55-9aca-0a254ea6af8e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:34.690804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:34.691812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:34.692336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:34.692846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:34.696331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:34.698728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:34.701204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 62/131] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:34 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:34.702243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:34.727202 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94abc3-bc83-4e76-8f9c-dcd83ac6f64f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:34.779509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:34.802530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:34.804471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:34.804850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:34.805144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/97726e6afd31490a96b8f997adddff27/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:34.808574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] RBAC: Authorizing `identity:create_grant(project_id=d90586d8e9094e65ad00907c4b6c8ddd, user_id=97726e6afd31490a96b8f997adddff27, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:34.810043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:34.830973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 70/132] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:34 2022] PUT /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/97726e6afd31490a96b8f997adddff27/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:34.831780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:34.919638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e5c5f-029f-447a-9d0e-30b5602f415e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:34.978716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:35.007081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:35.007805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.008212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.008426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.009211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:35.009522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:35.010942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:35.030986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 63/133] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:35.037812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:35.045540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6ce1-55cb-46b5-9c6a-cc7bf97b91de None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:35.084046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ba1ea-221d-4687-96da-329feb5cce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:35.085276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ba1ea-221d-4687-96da-329feb5cce82 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.085702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ba1ea-221d-4687-96da-329feb5cce82 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.086195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ba1ea-221d-4687-96da-329feb5cce82 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/1829d1e77c554b12a2c6482e57c9fe41/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.088796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ba1ea-221d-4687-96da-329feb5cce82 None admin] RBAC: Authorizing `identity:create_grant(project_id=d90586d8e9094e65ad00907c4b6c8ddd, user_id=1829d1e77c554b12a2c6482e57c9fe41, role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:35.090529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ba1ea-221d-4687-96da-329feb5cce82 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:35.111414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 71/134] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:34 2022] PUT /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/1829d1e77c554b12a2c6482e57c9fe41/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:35.114027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4042-e596-4e78-b7c8-03acf9f2eb6c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.116136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4042-e596-4e78-b7c8-03acf9f2eb6c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.116864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4042-e596-4e78-b7c8-03acf9f2eb6c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.120924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 72/135] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:35.128694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1f53d-ec65-44dc-8755-992c9649095a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.129345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1f53d-ec65-44dc-8755-992c9649095a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.129824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1f53d-ec65-44dc-8755-992c9649095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.157510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-43f1f53d-ec65-44dc-8755-992c9649095a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:35.159066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-43f1f53d-ec65-44dc-8755-992c9649095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:35.234627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc419a2-8320-42fb-8656-9242f4ea5f06 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:35.237295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:35.243901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 73/136] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:35.245700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a225c-f67d-45c5-9f25-a8d6e85536ce None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.246221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a225c-f67d-45c5-9f25-a8d6e85536ce None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.246508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a225c-f67d-45c5-9f25-a8d6e85536ce None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.247783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 74/137] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:35.254009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0b7da-922b-4b37-8174-00ed332d2384 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.254503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0b7da-922b-4b37-8174-00ed332d2384 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.254952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0b7da-922b-4b37-8174-00ed332d2384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.274334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-66d0b7da-922b-4b37-8174-00ed332d2384 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:35.275842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-66d0b7da-922b-4b37-8174-00ed332d2384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:35.282421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:35.283144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.283590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.283936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.285385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:35.289265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:35.291401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 64/138] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:35 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:35.292565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:35.381441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:35.433245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 75/139] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:35.436130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21921f-9b90-4e95-96e1-821938e37728 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:35.437417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0b7da-922b-4b37-8174-00ed332d2384 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:35.495601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:35.496680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.497178 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.497680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.518045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:35.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:35.523758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:35.524525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 65/140] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:35.526650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:35.592667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:35.603094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0b7da-922b-4b37-8174-00ed332d2384 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:35.665601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:35.666286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.666579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.666859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] PATH_INFO: `/v3/projects/d02a5a36a45a44e8b905905893ed0361` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.668623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] RBAC: Authorizing `identity:get_project(project_id=d02a5a36a45a44e8b905905893ed0361)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:35.670716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:35.673208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 76/141] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:35 2022] GET /identity/v3/projects/d02a5a36a45a44e8b905905893ed0361 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:35.679162 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:35.717352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b796d5-55c0-409c-b18b-f4355fb7fbcc None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:35.777245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:35.791124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:35.792627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:35.793196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:35.793687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:35.795150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:35.796905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:35.799529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 66/142] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:35.800468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:35.921996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b217d-4577-4726-9e0d-d24eca157914 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:35.958497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:35.967517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c371c2-230e-4286-9345-ca8d7cab2dce None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:36.003034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:36.004802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.007382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.007879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.008841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:36.009364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:36.010962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:36.020151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 77/143] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:36.021656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:36.049598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:36.050299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.050590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.050891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] PATH_INFO: `/v3/projects/d02a5a36a45a44e8b905905893ed0361/users/1829d1e77c554b12a2c6482e57c9fe41/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.057915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] RBAC: Authorizing `identity:create_grant(project_id=d02a5a36a45a44e8b905905893ed0361, user_id=1829d1e77c554b12a2c6482e57c9fe41, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:36.059395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:36.085400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 67/144] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:35 2022] PUT /identity/v3/projects/d02a5a36a45a44e8b905905893ed0361/users/1829d1e77c554b12a2c6482e57c9fe41/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:36.086318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:36.178794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d00aa-9059-4b0d-9192-5df22f06cbbf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:36.186923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:36.242410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:36.243430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.244009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.244494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.245495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:36.247176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:36.288758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 78/145] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:12:36 2022] POST /identity/v3/users => generated 283 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:36.338777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3201a-69de-44d7-81e3-02722386042e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:36.387777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:36.388465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.388788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.389068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] PATH_INFO: `/v3/users/8b67a047eae74976b581ea18ee84c963` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.392649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] RBAC: Authorizing `identity:get_user(user_id=8b67a047eae74976b581ea18ee84c963)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:36.394243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:36.397378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 68/146] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:36 2022] GET /identity/v3/users/8b67a047eae74976b581ea18ee84c963 => generated 289 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:36.402352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:36.563159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1662c9f-526a-4cfc-846d-8ff92a2e9a2a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:36.605090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a72bc-43ce-4338-9d32-56201be47fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:36.606103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a72bc-43ce-4338-9d32-56201be47fea None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.606603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a72bc-43ce-4338-9d32-56201be47fea None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.607088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a72bc-43ce-4338-9d32-56201be47fea None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.608478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a72bc-43ce-4338-9d32-56201be47fea None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:36.610181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a72bc-43ce-4338-9d32-56201be47fea None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:36.612206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 79/147] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:36 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:36.620012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:36.625447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970f525-5216-4149-a8f3-067a159dd505 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.626068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970f525-5216-4149-a8f3-067a159dd505 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.626566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970f525-5216-4149-a8f3-067a159dd505 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.628200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 80/148] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:36.641517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b315a42-8762-4f0a-ad28-2324260de711 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.642228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b315a42-8762-4f0a-ad28-2324260de711 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.642778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b315a42-8762-4f0a-ad28-2324260de711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.663467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9b315a42-8762-4f0a-ad28-2324260de711 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:36.665119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9b315a42-8762-4f0a-ad28-2324260de711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:36.681955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:36.699066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:36.743823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 81/149] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:36.785924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b315a42-8762-4f0a-ad28-2324260de711 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:36.832199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f34037-c258-4ef4-b484-fa09c7df1410 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:36.902181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:36.902884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.903318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:36.903610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/8b67a047eae74976b581ea18ee84c963/roles/f051e915d2d1479ba724a09298df84b7` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:36.907484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d90586d8e9094e65ad00907c4b6c8ddd, user_id=8b67a047eae74976b581ea18ee84c963, role_id=f051e915d2d1479ba724a09298df84b7)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:36.909176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:36.932470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 69/150] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:36 2022] PUT /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/users/8b67a047eae74976b581ea18ee84c963/roles/f051e915d2d1479ba724a09298df84b7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:36.954142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b315a42-8762-4f0a-ad28-2324260de711 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:36.998838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:36.999494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:36.999780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:37.000061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:37.021544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:37.027515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:37.028415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:37.029187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 82/151] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:37.034880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:37.157274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:37.179736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e650-9b0d-485f-b33c-fd6012b749b8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:37.222698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:37.223336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:37.223621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:37.223904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:37.224836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:37.225230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:37.226629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:37.238589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 70/152] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:37.245091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:37.278968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:37.372897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7835adc-f809-404c-baa4-b8b467a11bfc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:37.418346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918602d0-5318-4343-b8ed-5cc9162e8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:37.418940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918602d0-5318-4343-b8ed-5cc9162e8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:37.419219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918602d0-5318-4343-b8ed-5cc9162e8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:37.419507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918602d0-5318-4343-b8ed-5cc9162e8ba9 None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:37.420790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918602d0-5318-4343-b8ed-5cc9162e8ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:37.422333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918602d0-5318-4343-b8ed-5cc9162e8ba9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:37.424489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 83/153] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:37 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:37.430446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:37.557011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50657c-f1fa-4115-81fc-30d5a6890648 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:37.557339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50657c-f1fa-4115-81fc-30d5a6890648 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:37.557571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50657c-f1fa-4115-81fc-30d5a6890648 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:37.558965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 84/154] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:37.564139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e0d3e-c5dd-431a-92b0-8ab96018d3d0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:37.564435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e0d3e-c5dd-431a-92b0-8ab96018d3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:37.564744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e0d3e-c5dd-431a-92b0-8ab96018d3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:37.575761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9eede7-379c-4b68-a8e7-76046a68cad8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:37.597608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-db9e0d3e-c5dd-431a-92b0-8ab96018d3d0 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:37.598813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-db9e0d3e-c5dd-431a-92b0-8ab96018d3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:37.615141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:37.616125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:37.616654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:37.617122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:37.652387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:37.715848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 85/155] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 29 06:12:37.754932 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e0d3e-c5dd-431a-92b0-8ab96018d3d0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:37.764158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:37.765690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:37.788224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:37.918585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e0d3e-c5dd-431a-92b0-8ab96018d3d0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:37.969544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:37.970310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:37.970596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:37.970870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:37.986172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:37.993830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:37.994907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:37.995739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 86/156] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:38.001266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:38.027469 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 71/157] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:38.038168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:38.128236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:38.147687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:38.191587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973535f0-cf84-40ea-9b3c-bba32b1f634d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:38.230841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d26a55-a221-41dc-82dc-e59508c6b6fd None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:38.257216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259c928-ca66-4a4a-82f2-75db01f90c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:38.257771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c928-ca66-4a4a-82f2-75db01f90c0b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:38.258018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c928-ca66-4a4a-82f2-75db01f90c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:38.258299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c928-ca66-4a4a-82f2-75db01f90c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:38.259040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259c928-ca66-4a4a-82f2-75db01f90c0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:38.259350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259c928-ca66-4a4a-82f2-75db01f90c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:38.262004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259c928-ca66-4a4a-82f2-75db01f90c0b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:38.276313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 87/158] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:38.279155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5af7-d34c-4cbb-b4fa-177af4ceaf07 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:38.279443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5af7-d34c-4cbb-b4fa-177af4ceaf07 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:38.279728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5af7-d34c-4cbb-b4fa-177af4ceaf07 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:38.281114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 88/159] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:38.285876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5af7-d34c-4cbb-b4fa-177af4ceaf07 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:38.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb088401-e044-4e42-96d9-ab2a0dad6a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:38.303336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb088401-e044-4e42-96d9-ab2a0dad6a1d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:38.303543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb088401-e044-4e42-96d9-ab2a0dad6a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:38.303775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb088401-e044-4e42-96d9-ab2a0dad6a1d None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:38.305026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb088401-e044-4e42-96d9-ab2a0dad6a1d None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:38.306488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb088401-e044-4e42-96d9-ab2a0dad6a1d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:38.309390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 72/160] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:38 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:38.316522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317e98d-f3c2-44c3-b8d1-5bb3153c8e9f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:38.316840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317e98d-f3c2-44c3-b8d1-5bb3153c8e9f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:38.317120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317e98d-f3c2-44c3-b8d1-5bb3153c8e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:38.351361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3317e98d-f3c2-44c3-b8d1-5bb3153c8e9f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:38.352603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3317e98d-f3c2-44c3-b8d1-5bb3153c8e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:38.491225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 73/161] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:38.492453 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317e98d-f3c2-44c3-b8d1-5bb3153c8e9f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:38.498113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5af7-d34c-4cbb-b4fa-177af4ceaf07 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:38.564617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:38.565275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:38.565562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:38.565842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] PATH_INFO: `/v3/users/97726e6afd31490a96b8f997adddff27` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:38.567095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] RBAC: Authorizing `identity:get_user(user_id=97726e6afd31490a96b8f997adddff27)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:38.568627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:38.570238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 89/162] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:38 2022] GET /identity/v3/users/97726e6afd31490a96b8f997adddff27 => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:38.572336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:38.578282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:38.594180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:38.675350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317e98d-f3c2-44c3-b8d1-5bb3153c8e9f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:38.738851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b053b-b915-45f6-8ec0-2c11368a0bab None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:38.783631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:38.784182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:38.784463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:38.784766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:38.785558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=1829d1e77c554b12a2c6482e57c9fe41) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:38.785864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:38.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:38.800868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 74/163] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:38 2022] GET /identity/v3/role_assignments?user.id=1829d1e77c554b12a2c6482e57c9fe41&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:38.801694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:38.804691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:38.805640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:38.805931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:38.806221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:38.832773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:38.844783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:38.845723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:38.846484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 90/164] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:12:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:38.853365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:38.985325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:39.057109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8d69-3a5b-4ac8-a300-78a10d67a79a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:39.062588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446ebdc-d9ee-4577-a938-00c07ff3ea96 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:39.081095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:39.137293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:39.138008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:39.138302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:39.138586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:39.139384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:39.139703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:39.141319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:39.142982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:39.143862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:39.144354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:39.144861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:39.161574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 91/165] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:12:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:39.167507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:39.281766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:39.284131 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:39.442884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d22aa-a21a-4bb8-a9a4-2e3591e8aa7a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:39.468332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 75/166] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:39.475405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:39.500928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:39.522190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:39.538522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:39.539179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:39.539464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:39.539740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:39.540966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:39.542655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:39.544292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 92/167] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:39 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:39.552563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:39.692354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eda732-a4f1-4870-8725-7cda0fa0230d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:39.695417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a1d4-b689-4932-a52e-4749092be321 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:39.757392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:39.758496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:39.758994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:39.759467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:39.760986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:39.762209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433ccc1-ffe6-4d2f-9bdf-8a99d4e4f1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:39.762851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433ccc1-ffe6-4d2f-9bdf-8a99d4e4f1bb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:39.763135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433ccc1-ffe6-4d2f-9bdf-8a99d4e4f1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:39.763369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:39.763467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433ccc1-ffe6-4d2f-9bdf-8a99d4e4f1bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:39.765783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 76/168] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:39 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:39.789638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:39.863893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:39.868563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433ccc1-ffe6-4d2f-9bdf-8a99d4e4f1bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:39.868563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433ccc1-ffe6-4d2f-9bdf-8a99d4e4f1bb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:39.926900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:39.959711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f2ef4-8a46-47ab-83c0-8be33f82f5a7 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:40.017967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:40.019011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:40.019501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:40.019982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:40.021058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=97726e6afd31490a96b8f997adddff27) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:40.022215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:40.023886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:40.036774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 77/169] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:39 2022] GET /identity/v3/role_assignments?user.id=97726e6afd31490a96b8f997adddff27&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:40.044238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 93/170] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:40.050109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:40.099874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba116d47-6993-4545-8bee-073370277104 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:40.100668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba116d47-6993-4545-8bee-073370277104 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:40.101154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba116d47-6993-4545-8bee-073370277104 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:40.102777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 94/171] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:40.107990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921004b-0e58-4d73-b13d-1f97106f1bd7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:40.108612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921004b-0e58-4d73-b13d-1f97106f1bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:40.109125 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921004b-0e58-4d73-b13d-1f97106f1bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:40.148995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e921004b-0e58-4d73-b13d-1f97106f1bd7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:40.150562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e921004b-0e58-4d73-b13d-1f97106f1bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:40.182810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaae75c-cd9b-49aa-8c2e-81142f760bd0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:40.227154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:40.227741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:40.228029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:40.228306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:40.230468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:40.231882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:40.233703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 78/172] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:40 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:40.246007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:40.272752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:40.313794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 95/173] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 29 06:12:40.384624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921004b-0e58-4d73-b13d-1f97106f1bd7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:40.445788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:40.450261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452403fb-5c77-4406-a72c-96fa292bead5 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:40.520229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:40.520844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:40.521133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:40.521942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:40.522838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] RBAC: Adding query filter params (role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=1829d1e77c554b12a2c6482e57c9fe41) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:40.523283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:40.524773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:40.531771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 79/174] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:40 2022] GET /identity/v3/role_assignments?user.id=1829d1e77c554b12a2c6482e57c9fe41&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:40.603352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921004b-0e58-4d73-b13d-1f97106f1bd7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:40.652192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:40.652878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:40.653166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:40.653448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:40.673600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:40.678911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:40.679762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:40.680992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 96/175] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:40.686137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:40.799423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:40.834153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f18a6e-5434-42bf-b147-769c6fd61176 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:40.878676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d09e6-9cd8-428b-9a2f-3fe8760aaca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:40.879323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d09e6-9cd8-428b-9a2f-3fe8760aaca3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:40.879603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d09e6-9cd8-428b-9a2f-3fe8760aaca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:40.879886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d09e6-9cd8-428b-9a2f-3fe8760aaca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:40.881184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d09e6-9cd8-428b-9a2f-3fe8760aaca3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:40.881638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d09e6-9cd8-428b-9a2f-3fe8760aaca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:40.883239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d09e6-9cd8-428b-9a2f-3fe8760aaca3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:40.892801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 80/176] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:40.900624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.050914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:41.070189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcffa92-a45c-4db8-9b1b-115d94937fa0 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.070503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcffa92-a45c-4db8-9b1b-115d94937fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.070761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcffa92-a45c-4db8-9b1b-115d94937fa0 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.072397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 81/177] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:41.077317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecfbb4-70ea-4cf7-a021-59c2c66db1c7 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.077607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecfbb4-70ea-4cf7-a021-59c2c66db1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.077890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecfbb4-70ea-4cf7-a021-59c2c66db1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.081096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33294-a790-4d27-8380-5447fcfa6272 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.103820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-78ecfbb4-70ea-4cf7-a021-59c2c66db1c7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:41.105067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-78ecfbb4-70ea-4cf7-a021-59c2c66db1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:41.139842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e983c09-4d73-41de-b2ac-62c3a555a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:41.140653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e983c09-4d73-41de-b2ac-62c3a555a064 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.140940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e983c09-4d73-41de-b2ac-62c3a555a064 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.141220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e983c09-4d73-41de-b2ac-62c3a555a064 None admin] PATH_INFO: `/v3/users/1829d1e77c554b12a2c6482e57c9fe41` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.142484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e983c09-4d73-41de-b2ac-62c3a555a064 None admin] RBAC: Authorizing `identity:get_user(user_id=1829d1e77c554b12a2c6482e57c9fe41)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:41.144201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e983c09-4d73-41de-b2ac-62c3a555a064 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:41.146286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 97/178] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:40 2022] GET /identity/v3/users/1829d1e77c554b12a2c6482e57c9fe41 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:41.147695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed51ba-60bf-4d39-a913-14a7fe7a80bf None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.147986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed51ba-60bf-4d39-a913-14a7fe7a80bf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.148472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed51ba-60bf-4d39-a913-14a7fe7a80bf None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.149923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 98/179] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:41.154659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed51ba-60bf-4d39-a913-14a7fe7a80bf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.197589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 82/180] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:41.198952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6eadc-2bf4-41fc-abec-cea0044f68cd None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.199246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6eadc-2bf4-41fc-abec-cea0044f68cd None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.199527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6eadc-2bf4-41fc-abec-cea0044f68cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.229589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c3e6eadc-2bf4-41fc-abec-cea0044f68cd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:41.231019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c3e6eadc-2bf4-41fc-abec-cea0044f68cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:41.297069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed51ba-60bf-4d39-a913-14a7fe7a80bf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.299857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:41.308206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 83/181] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:41.309396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6eadc-2bf4-41fc-abec-cea0044f68cd None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.333698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:41.335178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.335469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.335746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.360625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:41.405203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:41.406715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:41.412921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6eadc-2bf4-41fc-abec-cea0044f68cd None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.446566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88416-b2f9-469a-acac-2addccda9ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:41.447208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88416-b2f9-469a-acac-2addccda9ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.447493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88416-b2f9-469a-acac-2addccda9ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.447788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88416-b2f9-469a-acac-2addccda9ad2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e88416-b2f9-469a-acac-2addccda9ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-10e88416-b2f9-469a-acac-2addccda9ad2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:41.463038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:41.464055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:41.464877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 84/182] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:41.465514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cc4cd-e0a5-4466-941c-86dd0d941f09 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.465626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cc4cd-e0a5-4466-941c-86dd0d941f09 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.465968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cc4cd-e0a5-4466-941c-86dd0d941f09 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.467221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 85/183] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:41.471179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a51c0-4d25-4f49-ae62-d01047b93b7a None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.471179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a51c0-4d25-4f49-ae62-d01047b93b7a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.471179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a51c0-4d25-4f49-ae62-d01047b93b7a None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.471179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 86/184] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:41.471429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a51c0-4d25-4f49-ae62-d01047b93b7a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.513908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 99/185] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:41.514758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.531501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:41.576513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:41.577564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a51c0-4d25-4f49-ae62-d01047b93b7a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.612229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb25158-c4bd-492b-abb7-8900cccc1cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:41.612905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb25158-c4bd-492b-abb7-8900cccc1cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.613197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb25158-c4bd-492b-abb7-8900cccc1cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.613474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb25158-c4bd-492b-abb7-8900cccc1cc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.620722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4bf03-de56-4760-a224-cbc1f3e9fe54 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.624526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb25158-c4bd-492b-abb7-8900cccc1cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-3fb25158-c4bd-492b-abb7-8900cccc1cc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:41.629439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:41.630299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 87/186] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:41.632504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd3fa5-dc0d-4746-aceb-572753b01bdd None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.633370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd3fa5-dc0d-4746-aceb-572753b01bdd None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.633370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd3fa5-dc0d-4746-aceb-572753b01bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.650593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c8fd3fa5-dc0d-4746-aceb-572753b01bdd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:41.651788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c8fd3fa5-dc0d-4746-aceb-572753b01bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:41.659837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9c123-e13a-42a5-abee-75454d33e40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:41.660459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9c123-e13a-42a5-abee-75454d33e40e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.660781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9c123-e13a-42a5-abee-75454d33e40e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.661060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9c123-e13a-42a5-abee-75454d33e40e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.661800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9c123-e13a-42a5-abee-75454d33e40e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:41.662142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9c123-e13a-42a5-abee-75454d33e40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:41.663681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9c123-e13a-42a5-abee-75454d33e40e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:41.672185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 100/187] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:41.673480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d66b2-b7a2-4b83-a2c7-7755e67fb50b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.673772 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d66b2-b7a2-4b83-a2c7-7755e67fb50b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.674050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d66b2-b7a2-4b83-a2c7-7755e67fb50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.691853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f07d66b2-b7a2-4b83-a2c7-7755e67fb50b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:41.693658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f07d66b2-b7a2-4b83-a2c7-7755e67fb50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:41.717209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 88/188] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:41.718049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd3fa5-dc0d-4746-aceb-572753b01bdd None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.765235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:41.772308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 101/189] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:41.773270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d66b2-b7a2-4b83-a2c7-7755e67fb50b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.822922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:41.829726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd3fa5-dc0d-4746-aceb-572753b01bdd None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.863514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:41.864277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.864585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.865378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] PATH_INFO: `/v3/projects/d02a5a36a45a44e8b905905893ed0361` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.866548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] RBAC: Authorizing `identity:get_project(project_id=d02a5a36a45a44e8b905905893ed0361)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:41.867989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:41.869705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 89/190] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/projects/d02a5a36a45a44e8b905905893ed0361 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:41.870492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.872977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d66b2-b7a2-4b83-a2c7-7755e67fb50b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.908686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:41.909314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:41.909630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:41.909910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:41.910631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:41.910936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:41.912360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:41.920517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 102/191] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:41.921333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:41.974252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25621a-7a56-434d-b331-b59dca7b4b66 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:41.987423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.008392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.009068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.009362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.009638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] PATH_INFO: `/v3/users/8b67a047eae74976b581ea18ee84c963` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.010807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] RBAC: Authorizing `identity:get_user(user_id=8b67a047eae74976b581ea18ee84c963)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.012250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.013907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 90/192] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/users/8b67a047eae74976b581ea18ee84c963 => generated 289 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:42.014667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.021254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7c1b0-9d10-43d7-8187-bbce58f4597c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.031671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.056404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.057071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.057353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.057626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.058757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.060189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.061830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 103/193] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:42.062581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.120001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b674c7-43d1-4eb6-8148-e975247f4d95 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.152472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.153141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.153423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.153698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.160250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dd914-d939-44bf-a92c-be638161b50b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.164329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:42.169718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:42.170594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:42.171375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 91/194] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:42.172010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.196062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.196719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.197008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.197289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.198070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=d02a5a36a45a44e8b905905893ed0361, user.id=1829d1e77c554b12a2c6482e57c9fe41) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:42.198388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.199817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.205907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 104/195] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/role_assignments?user.id=1829d1e77c554b12a2c6482e57c9fe41&scope.project.id=d02a5a36a45a44e8b905905893ed0361&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:42.206685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.219814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.262588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.299040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327c739-672f-4ccb-acda-9a18ae92d0c5 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.327451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da47b5-499b-4b29-86fe-03154f89fc00 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.337135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.337797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.338081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.338355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] PATH_INFO: `/v3/users/84ee2bd3bcdc4950a4fd029642d833a3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.354904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] RBAC: Authorizing `identity:get_user(user_id=84ee2bd3bcdc4950a4fd029642d833a3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.356710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.358176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 92/196] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/users/84ee2bd3bcdc4950a4fd029642d833a3 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:42.358176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.374593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.375159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.375449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.375722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.452435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.453924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.470788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.474106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c259-ceb2-45c4-a75a-d25aec15143b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.495842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.510931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.511551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.511828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.512096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.512797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.514222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.534630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 93/197] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:12:42 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:42.535566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.572600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 105/198] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:42.572600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.664600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e00a6b-bd6e-4f8c-ac42-f55bc8f04cc0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.684859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b44f6-7d84-4882-9539-b60490f736ad None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.703194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.703611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.703895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.704173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.706765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:42.706765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.706967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.716252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 94/199] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:42.717211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.736197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.736872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.737164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.737516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.740916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.756243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:42.835288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.836857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.858696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc5df7-2017-4cd9-bf2a-f32e749d731b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:42.898133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:42.899056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:42.899691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:42.900111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:42.901660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:42.903414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:42.905558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 95/200] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:42.906721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:42.962214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 106/201] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:42.963088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:43.006534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:43.015858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:43.025393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adc316-0223-4e61-bbfe-42c429d16054 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:43.061570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:43.062467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:43.063168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:43.063610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] PATH_INFO: `/v3/users/84ee2bd3bcdc4950a4fd029642d833a3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:43.064901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] RBAC: Authorizing `identity:get_user(user_id=84ee2bd3bcdc4950a4fd029642d833a3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:43.066847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:43.069980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 96/202] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/users/84ee2bd3bcdc4950a4fd029642d833a3 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:43.070827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:43.081631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd43435-237c-4697-a38f-56e27be655af None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:43.133900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:43.135168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:43.135951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:43.136726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:43.138498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] RBAC: Adding query filter params (role.id=f051e915d2d1479ba724a09298df84b7, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd, user.id=8b67a047eae74976b581ea18ee84c963) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:43.139425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:43.141656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:43.150753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 107/203] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:42 2022] GET /identity/v3/role_assignments?user.id=8b67a047eae74976b581ea18ee84c963&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=f051e915d2d1479ba724a09298df84b7 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:43.152040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:43.223729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd39969-fec6-4ba3-ac6a-fa0533099f2a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:43.316307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:43.317471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:43.319225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:43.320173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:43.323011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:43.326516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:43.329666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 97/204] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:43 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:43.337513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:43.374962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509168be-7f12-4657-912f-1f08260e8c07 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:43.388084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:43.410368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:43.428652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:43.429293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:43.429578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:43.429856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:43.515247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d2e9-2e19-4041-8772-37ce2b3ccaa0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:43.539460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:43.542924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:43.578225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:43.579248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:43.579769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:43.580492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:43.581538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=84ee2bd3bcdc4950a4fd029642d833a3) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:43.582156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:43.584700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:43.591918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 98/205] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:43 2022] GET /identity/v3/role_assignments?user.id=84ee2bd3bcdc4950a4fd029642d833a3&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:43.690326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 108/206] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:43.697802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:43.801274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:43.818596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44aba9-c620-4004-b803-bddc6e3eacfa None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:43.857661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63a3be-3533-41bb-879f-a139abb059d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:43.858322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63a3be-3533-41bb-879f-a139abb059d0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:43.858680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63a3be-3533-41bb-879f-a139abb059d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:43.859002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63a3be-3533-41bb-879f-a139abb059d0 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:43.860238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63a3be-3533-41bb-879f-a139abb059d0 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:43.861726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63a3be-3533-41bb-879f-a139abb059d0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:43.863416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 99/207] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:43 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:43.869958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:43.880966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:43.993160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4807e-406b-4b86-b5fc-f640fc29b016 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:44.032058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5915244-c266-4da8-b105-18b8265389f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:44.033203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915244-c266-4da8-b105-18b8265389f2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:44.033591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915244-c266-4da8-b105-18b8265389f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:44.033971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915244-c266-4da8-b105-18b8265389f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:44.034936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915244-c266-4da8-b105-18b8265389f2 None admin] RBAC: Adding query filter params (role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=84ee2bd3bcdc4950a4fd029642d833a3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:44.035428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915244-c266-4da8-b105-18b8265389f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:44.038169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915244-c266-4da8-b105-18b8265389f2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:44.044988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 109/208] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:43 2022] GET /identity/v3/role_assignments?user.id=84ee2bd3bcdc4950a4fd029642d833a3&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:44.934733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd6b5-219b-4809-a30c-f0e49f357859 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:44.935762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd6b5-219b-4809-a30c-f0e49f357859 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:44.936269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd6b5-219b-4809-a30c-f0e49f357859 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:44.937867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 100/209] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:44.943568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef3fde-edc6-4be0-9213-14068207a997 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:44.944402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef3fde-edc6-4be0-9213-14068207a997 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:44.944911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef3fde-edc6-4be0-9213-14068207a997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:44.964968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-54ef3fde-edc6-4be0-9213-14068207a997 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:44.966468 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-54ef3fde-edc6-4be0-9213-14068207a997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:44.982377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc38ce-2f90-4d96-8d69-7f98f936a25b None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:44.982652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc38ce-2f90-4d96-8d69-7f98f936a25b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:44.982874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc38ce-2f90-4d96-8d69-7f98f936a25b None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:44.984488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 101/210] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:44.989399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06225-8d19-4916-bd08-c6e0757212c2 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:44.989694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06225-8d19-4916-bd08-c6e0757212c2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:44.990113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06225-8d19-4916-bd08-c6e0757212c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.013347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f3a06225-8d19-4916-bd08-c6e0757212c2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:45.014557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f3a06225-8d19-4916-bd08-c6e0757212c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:45.044451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:45.052928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 110/211] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:45.075624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef3fde-edc6-4be0-9213-14068207a997 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:45.097335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 102/212] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:45.124510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a06225-8d19-4916-bd08-c6e0757212c2 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:45.194075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef3fde-edc6-4be0-9213-14068207a997 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:45.232886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:45.233878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.234398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:45.234946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.236068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.250394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:45.252595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a06225-8d19-4916-bd08-c6e0757212c2 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:45.256386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:45.257439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:45.258235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:45.261489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 111/213] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:45.267804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:45.295614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:45.296809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.297312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.297802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.299323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:45.301008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:45.308597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 103/214] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:45.309766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:45.345262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:45.391948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c022f-be4f-4244-9dbf-8337ca078d22 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:45.434321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:45.435389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.435902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.436397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.437355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:45.437865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:45.438193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad0639-2316-4646-9f0d-69b53dfcf679 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:45.439848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:45.449579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 112/215] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:45.455861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:45.480530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:45.481532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.482025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.482528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.483531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:45.485221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:45.524903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 104/216] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:12:45 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:45.589617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a25fc6-113a-4bea-804a-e21afdd53fd6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:45.616672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:45.628947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb90016-b3fd-4f54-87c8-446c3a9b2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:45.629608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb90016-b3fd-4f54-87c8-446c3a9b2738 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.629894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb90016-b3fd-4f54-87c8-446c3a9b2738 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.630172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb90016-b3fd-4f54-87c8-446c3a9b2738 None admin] PATH_INFO: `/v3/users/b37c9fb8dd9c4b21a6dfcc0ff42062da` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.646307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb90016-b3fd-4f54-87c8-446c3a9b2738 None admin] RBAC: Authorizing `identity:get_user(user_id=b37c9fb8dd9c4b21a6dfcc0ff42062da)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:45.647924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb90016-b3fd-4f54-87c8-446c3a9b2738 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:45.649946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 113/217] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:45 2022] GET /identity/v3/users/b37c9fb8dd9c4b21a6dfcc0ff42062da => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:45.655571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:45.673144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:45.754068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fdc70-313c-409f-84d7-0de6c70df098 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.754677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fdc70-313c-409f-84d7-0de6c70df098 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.755163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fdc70-313c-409f-84d7-0de6c70df098 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.757421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 114/218] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:45.764356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79a19-5f1e-466f-8a48-abf33a88a86d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.764940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79a19-5f1e-466f-8a48-abf33a88a86d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.765446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79a19-5f1e-466f-8a48-abf33a88a86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.790771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc7e4-e46d-40cd-805a-3cc070bab2ab None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:45.793025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-47d79a19-5f1e-466f-8a48-abf33a88a86d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:45.794229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-47d79a19-5f1e-466f-8a48-abf33a88a86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:45.836513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:45.837867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:45.838565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:45.839125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:45.887461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 115/219] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:45.910588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79a19-5f1e-466f-8a48-abf33a88a86d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:45.924627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:45.926114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:45.939011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:46.040918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79a19-5f1e-466f-8a48-abf33a88a86d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.047717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 105/220] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:46.057544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.060897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:46.093562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77c7ae-dcba-4cd0-8b4b-b1a11ecf1efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.094337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77c7ae-dcba-4cd0-8b4b-b1a11ecf1efe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.094693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77c7ae-dcba-4cd0-8b4b-b1a11ecf1efe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.095052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77c7ae-dcba-4cd0-8b4b-b1a11ecf1efe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.118705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77c7ae-dcba-4cd0-8b4b-b1a11ecf1efe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ef77c7ae-dcba-4cd0-8b4b-b1a11ecf1efe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:46.123854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:46.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:46.125565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:46.141168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 116/221] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:46.142643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc46f21-1e45-48c0-bd2e-e162ce866456 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.142941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc46f21-1e45-48c0-bd2e-e162ce866456 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.143226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc46f21-1e45-48c0-bd2e-e162ce866456 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.144560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 117/222] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:46.147575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc46f21-1e45-48c0-bd2e-e162ce866456 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.178957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaebe93-b29f-430e-a0fe-ee9c581116aa None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.189756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:46.219516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81edab-c565-4c34-bac6-13de6cc814f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.220273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81edab-c565-4c34-bac6-13de6cc814f7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.220570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81edab-c565-4c34-bac6-13de6cc814f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.220852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81edab-c565-4c34-bac6-13de6cc814f7 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.222035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81edab-c565-4c34-bac6-13de6cc814f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:46.223786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81edab-c565-4c34-bac6-13de6cc814f7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:46.226286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 106/223] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:46 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:46.227485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945148b-ae6a-4d73-9cfe-db4833d5343d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.227771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945148b-ae6a-4d73-9cfe-db4833d5343d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.228044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945148b-ae6a-4d73-9cfe-db4833d5343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.247325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5945148b-ae6a-4d73-9cfe-db4833d5343d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:46.248638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5945148b-ae6a-4d73-9cfe-db4833d5343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:46.261639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc46f21-1e45-48c0-bd2e-e162ce866456 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.298517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.299070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.299357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.299639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.300366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:46.300709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:46.302103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:46.315624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 118/224] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:46.316564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.324509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 107/225] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:46.325444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945148b-ae6a-4d73-9cfe-db4833d5343d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.350806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:46.423899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bb2de-dc4c-4e3d-b9f5-b4014f44a991 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.438068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945148b-ae6a-4d73-9cfe-db4833d5343d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.446853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:46.460205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.460870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.461156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.461438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.462237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=b37c9fb8dd9c4b21a6dfcc0ff42062da) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:46.462552 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:46.464005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:46.469831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 119/226] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:46 2022] GET /identity/v3/role_assignments?user.id=b37c9fb8dd9c4b21a6dfcc0ff42062da&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:46.477491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.481499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.482163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.482444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.482721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] PATH_INFO: `/v3/users/84ee2bd3bcdc4950a4fd029642d833a3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.483891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] RBAC: Authorizing `identity:get_user(user_id=84ee2bd3bcdc4950a4fd029642d833a3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:46.485354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:46.487379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 108/227] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:46 2022] GET /identity/v3/users/84ee2bd3bcdc4950a4fd029642d833a3 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:46.493831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.615561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe6ff-2618-4f5b-9434-5445be7d4b39 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.628903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:46.639519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e6014-4435-417d-8c1b-fe0f072d150f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.657260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.657996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.658277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.658608 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.659845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:46.661510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:46.663965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 109/228] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:46 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:46.669817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.682207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.682887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.683175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.683461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.695670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:46.701182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:46.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:46.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 120/229] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:12:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:46.710796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:46.728566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:46.790406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e967e-64f0-4ba2-b2c3-41f9e885b114 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.828024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.828758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.829052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.829455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/84ee2bd3bcdc4950a4fd029642d833a3/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.831847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] RBAC: Authorizing `identity:create_grant(project_id=b518d31d96ad4d21b04a249253c6a4d5, user_id=84ee2bd3bcdc4950a4fd029642d833a3, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:46.833468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:46.850471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 110/230] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:46 2022] PUT /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/84ee2bd3bcdc4950a4fd029642d833a3/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:12:46.912768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf806d8-1114-4558-8d0f-1f90cf0c6abd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:46.979518 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:46.980530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:46.981032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:46.981499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:46.982559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:46.983063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:46.984718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:47.001702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 121/231] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:12:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:47.008315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:47.131413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aa5c3-86be-4a8f-8aa4-7c2aea4010dd None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:47.133042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:47.171261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:47.171873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:47.172164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:47.172442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] PATH_INFO: `/v3/users/84ee2bd3bcdc4950a4fd029642d833a3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:47.173809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] RBAC: Authorizing `identity:get_user(user_id=84ee2bd3bcdc4950a4fd029642d833a3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:47.175251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:47.177050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 111/232] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:47 2022] GET /identity/v3/users/84ee2bd3bcdc4950a4fd029642d833a3 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:47.183099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:47.221591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:47.335845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc9d4-f291-4d3f-87ad-c7b30161ee4e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:47.386614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23eaed-1e17-4b7c-a6b2-353b71bb31eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:47.387274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23eaed-1e17-4b7c-a6b2-353b71bb31eb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:47.387562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23eaed-1e17-4b7c-a6b2-353b71bb31eb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:47.387842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23eaed-1e17-4b7c-a6b2-353b71bb31eb None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:47.389057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23eaed-1e17-4b7c-a6b2-353b71bb31eb None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:47.390520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23eaed-1e17-4b7c-a6b2-353b71bb31eb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:47.392195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 122/233] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:47 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:47.398250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:47.524107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e33144-4bbf-4144-9005-313dd08b0b42 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:47.560133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678dee1-d491-4b28-822a-d01ce1408268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:47.560907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678dee1-d491-4b28-822a-d01ce1408268 None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:47.561194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678dee1-d491-4b28-822a-d01ce1408268 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:47.561469 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678dee1-d491-4b28-822a-d01ce1408268 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/84ee2bd3bcdc4950a4fd029642d833a3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:47.563879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678dee1-d491-4b28-822a-d01ce1408268 None admin] RBAC: Authorizing `identity:create_grant(project_id=b518d31d96ad4d21b04a249253c6a4d5, user_id=84ee2bd3bcdc4950a4fd029642d833a3, role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:47.565359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678dee1-d491-4b28-822a-d01ce1408268 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:47.583655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 112/234] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:47 2022] PUT /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/84ee2bd3bcdc4950a4fd029642d833a3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:48.059263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605093b-50aa-4878-b4b0-4c6a82f7762b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.059546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605093b-50aa-4878-b4b0-4c6a82f7762b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.060101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605093b-50aa-4878-b4b0-4c6a82f7762b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.061693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 123/235] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:48.067033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62dded-159a-4508-84c1-e0e81df62ba4 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.068460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62dded-159a-4508-84c1-e0e81df62ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.069036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62dded-159a-4508-84c1-e0e81df62ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.089138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9b62dded-159a-4508-84c1-e0e81df62ba4 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:48.090716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9b62dded-159a-4508-84c1-e0e81df62ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:48.117663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:48.163376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 113/236] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:48.186245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605093b-50aa-4878-b4b0-4c6a82f7762b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:48.231718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bf2e-491a-45dc-9776-4f96973d9123 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.232372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bf2e-491a-45dc-9776-4f96973d9123 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.232877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bf2e-491a-45dc-9776-4f96973d9123 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.234686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 114/237] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:48.239590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ba43-7fc3-4b48-b400-f4ec7bf951da None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.240233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ba43-7fc3-4b48-b400-f4ec7bf951da None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.240844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ba43-7fc3-4b48-b400-f4ec7bf951da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.279619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3cb1ba43-7fc3-4b48-b400-f4ec7bf951da None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:48.281289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3cb1ba43-7fc3-4b48-b400-f4ec7bf951da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:48.282232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:48.301574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605093b-50aa-4878-b4b0-4c6a82f7762b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:48.342158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73061508-4153-4956-8856-93df7815410b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:48.343038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73061508-4153-4956-8856-93df7815410b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.343467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73061508-4153-4956-8856-93df7815410b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.343883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73061508-4153-4956-8856-93df7815410b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.355675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73061508-4153-4956-8856-93df7815410b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-73061508-4153-4956-8856-93df7815410b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:48.362504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:48.363532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:48.364465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:48.366436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 124/238] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:48.372395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73061508-4153-4956-8856-93df7815410b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:48.450519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 115/239] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:48.511151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73061508-4153-4956-8856-93df7815410b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:48.538673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ba43-7fc3-4b48-b400-f4ec7bf951da None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:48.554402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:48.555030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.555313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.555586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.556345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:48.556688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:48.558056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:48.567040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 125/240] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:48.572717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:48.585392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:48.686741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991da6d-f83e-4219-87ef-6e476d711abe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:48.725273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:48.725930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.726214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.726492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] PATH_INFO: `/v3/users/7c99098c4aac468ab030613c03e7ed18` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.741050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] RBAC: Authorizing `identity:get_user(user_id=7c99098c4aac468ab030613c03e7ed18)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:48.742617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:48.744455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 126/241] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:48 2022] GET /identity/v3/users/7c99098c4aac468ab030613c03e7ed18 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:48.749656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:48.761918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:48.781948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ba43-7fc3-4b48-b400-f4ec7bf951da None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:48.828976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:48.865364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39579cb5-665d-40ec-809a-8d6013f1f73b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:48.877248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:48.877915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.878201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.878482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.879666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:48.881688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:48.885595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 116/242] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:48.893426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:48.907528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:48.908154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:48.908477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:48.908824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:48.982741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:48.984144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:49.078666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:49.093073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 127/243] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 06:12:49.098497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:49.200378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cdf2a-bc45-49f3-bab3-346bf668f8af None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:49.211462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382fe00-08db-45c1-9917-f1e2a9b11298 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:49.245733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a1f4b-c7d4-4815-a1b0-fc62804e89a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:49.246376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a1f4b-c7d4-4815-a1b0-fc62804e89a0 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.246725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a1f4b-c7d4-4815-a1b0-fc62804e89a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.247066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a1f4b-c7d4-4815-a1b0-fc62804e89a0 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.247812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a1f4b-c7d4-4815-a1b0-fc62804e89a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:49.248829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d2349-83ae-442f-88db-7dbfa7cdb47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:49.249411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a1f4b-c7d4-4815-a1b0-fc62804e89a0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:49.249528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d2349-83ae-442f-88db-7dbfa7cdb47b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.249748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d2349-83ae-442f-88db-7dbfa7cdb47b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.250030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d2349-83ae-442f-88db-7dbfa7cdb47b None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.251209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d2349-83ae-442f-88db-7dbfa7cdb47b None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:49.252765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d2349-83ae-442f-88db-7dbfa7cdb47b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:49.254598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 128/244] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:49 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:49.256193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9877f-71c1-420f-b216-6b65246387f6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.256483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9877f-71c1-420f-b216-6b65246387f6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.257031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9877f-71c1-420f-b216-6b65246387f6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.258531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 129/245] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:49.261292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9877f-71c1-420f-b216-6b65246387f6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:49.281489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 117/246] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:12:48 2022] POST /identity/v3/users => generated 293 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:49.288085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b995c2e-6fb8-433e-b743-db969a517263 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.288388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b995c2e-6fb8-433e-b743-db969a517263 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.289217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b995c2e-6fb8-433e-b743-db969a517263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.324920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:49.335634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9b995c2e-6fb8-433e-b743-db969a517263 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:49.336946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9b995c2e-6fb8-433e-b743-db969a517263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:49.356855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:49.434558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9877f-71c1-420f-b216-6b65246387f6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:49.485721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 118/247] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:49.487032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458188d0-516e-49a9-88df-0e26e4c8ba29 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.487321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458188d0-516e-49a9-88df-0e26e4c8ba29 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.487596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458188d0-516e-49a9-88df-0e26e4c8ba29 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.490529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 119/248] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:49.491322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:49.492218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.492505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.492854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.493652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] RBAC: Adding query filter params (role.id=f051e915d2d1479ba724a09298df84b7, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=7c99098c4aac468ab030613c03e7ed18) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:49.494185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:49.495485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:49.495889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33defb92-a792-4999-a0f0-91b9b4bdab9e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.496365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33defb92-a792-4999-a0f0-91b9b4bdab9e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.496850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33defb92-a792-4999-a0f0-91b9b4bdab9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.501811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 130/249] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:49 2022] GET /identity/v3/role_assignments?user.id=7c99098c4aac468ab030613c03e7ed18&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=f051e915d2d1479ba724a09298df84b7 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:49.516283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:49.522794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-33defb92-a792-4999-a0f0-91b9b4bdab9e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:49.524026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-33defb92-a792-4999-a0f0-91b9b4bdab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:49.604741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 120/250] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:49.629567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33defb92-a792-4999-a0f0-91b9b4bdab9e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:49.638734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:49.643456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beae41-a086-4341-aa0e-5f9be79be62f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:49.681597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:49.682635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.683118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.683615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.695656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:49.701464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:49.702374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:49.703147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:49.705881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 131/251] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:49.712832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:49.723240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:49.752354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33defb92-a792-4999-a0f0-91b9b4bdab9e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:49.800569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:49.801416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.801807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.802292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.820168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:49.825842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:49.826936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:12:49.827918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:49.831929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 121/252] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:12:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:49.836944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab394a-0671-4fa2-8ec4-202ec6a2c38e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:49.838772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:49.879023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:49.880034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:49.881206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:49.881838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:49.882734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:49.883330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:49.885054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:49.894432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 132/253] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:49.901713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:49.931712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:49.981202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213b531-3093-4e1f-ae56-6241762ce7e0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:50.025317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:50.026491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.026951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.027397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.028471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:50.028972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:50.031248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:50.033912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:50.041124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22af020-4aa6-48fe-a22a-b1e83a12a76b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:50.043840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 122/254] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:50.056045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:50.087076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:50.087860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.088215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.088606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] PATH_INFO: `/v3/users/b37c9fb8dd9c4b21a6dfcc0ff42062da` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.089929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] RBAC: Authorizing `identity:get_user(user_id=b37c9fb8dd9c4b21a6dfcc0ff42062da)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:50.091781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:50.094782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 133/255] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:49 2022] GET /identity/v3/users/b37c9fb8dd9c4b21a6dfcc0ff42062da => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:50.101275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:50.220427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8b63-3fcc-4c29-a86f-f843428280e0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:50.260385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322f4fd-637f-4386-9f6d-ed07e3a498fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:50.261228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f4fd-637f-4386-9f6d-ed07e3a498fa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.261630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f4fd-637f-4386-9f6d-ed07e3a498fa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.261984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f4fd-637f-4386-9f6d-ed07e3a498fa None admin] PATH_INFO: `/v3/users/84ee2bd3bcdc4950a4fd029642d833a3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.263032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:50.264220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f4fd-637f-4386-9f6d-ed07e3a498fa None admin] RBAC: Authorizing `identity:get_user(user_id=84ee2bd3bcdc4950a4fd029642d833a3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:50.267744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f4fd-637f-4386-9f6d-ed07e3a498fa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:50.269207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 123/256] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:50 2022] GET /identity/v3/users/84ee2bd3bcdc4950a4fd029642d833a3 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:50.270204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36fa8-af16-46bc-b3b8-52f4e6797fb7 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.276904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36fa8-af16-46bc-b3b8-52f4e6797fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.277280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36fa8-af16-46bc-b3b8-52f4e6797fb7 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.279228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 124/257] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:50.280089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36fa8-af16-46bc-b3b8-52f4e6797fb7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:50.291060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652570d-888a-45a7-9c34-c7a251015e0c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:50.334963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a7377-716c-4671-9d06-7453b3fa490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:50.335912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a7377-716c-4671-9d06-7453b3fa490f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.337232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a7377-716c-4671-9d06-7453b3fa490f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.337711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a7377-716c-4671-9d06-7453b3fa490f None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.340368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a7377-716c-4671-9d06-7453b3fa490f None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:50.343086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a7377-716c-4671-9d06-7453b3fa490f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:50.348282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 134/258] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:50 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:50.354434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61163f9-e19d-438d-847a-d04b7bf07581 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.355444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61163f9-e19d-438d-847a-d04b7bf07581 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.357220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61163f9-e19d-438d-847a-d04b7bf07581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.391991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:50.402347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d61163f9-e19d-438d-847a-d04b7bf07581 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:50.403067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d61163f9-e19d-438d-847a-d04b7bf07581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:50.454527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36fa8-af16-46bc-b3b8-52f4e6797fb7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:50.520516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 135/259] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:50.526210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61163f9-e19d-438d-847a-d04b7bf07581 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:50.527042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:50.527875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.528346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.528810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.665092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:50.666632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:50.694608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:50.722719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61163f9-e19d-438d-847a-d04b7bf07581 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:50.777438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:50.796863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:50.797569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:50.797853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:50.798129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/b37c9fb8dd9c4b21a6dfcc0ff42062da/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:50.803084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] RBAC: Authorizing `identity:create_grant(project_id=b518d31d96ad4d21b04a249253c6a4d5, user_id=b37c9fb8dd9c4b21a6dfcc0ff42062da, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:50.804805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:50.827337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 136/260] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:50 2022] PUT /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/b37c9fb8dd9c4b21a6dfcc0ff42062da/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:50.827963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:50.869079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 125/261] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:50.874968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:51.082874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10ab9a-3653-4e03-94e7-2d46f5641286 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:51.132444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b828f3-df79-4961-9d9f-017a0bfe5102 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:51.135398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:51.136421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:51.136937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:51.137536 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:51.150145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:51.155732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:51.156658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:12:51.157462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:51.160186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 137/262] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:12:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:51.171106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:51.184423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:51.194952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:51.195628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:51.195913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:51.196192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:51.197521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:51.202060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:51.204092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 126/263] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:50 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:51.212689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:51.260256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:51.347167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df934f5-2d81-4a19-9fcc-2bee02baf96d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:51.354086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787b449-27f6-45d7-aedd-a3ab84fb3639 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:51.391394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:51.392028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:51.392318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:51.392617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:51.393418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=84ee2bd3bcdc4950a4fd029642d833a3) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:51.393737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:51.395185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:51.402999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 127/264] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:51 2022] GET /identity/v3/role_assignments?user.id=84ee2bd3bcdc4950a4fd029642d833a3&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:51.414712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:51.415709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:51.416235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:51.416758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:51.417698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:51.418213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:51.419984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:51.430676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 138/265] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:12:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:12:51.436297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:51.574514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:51.581726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287c40-bd0e-4c3b-8ae3-dee8ef8551bd None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:51.619029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff801bc-5b61-45e5-8d3f-97d025048d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:51.619616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff801bc-5b61-45e5-8d3f-97d025048d69 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:51.619904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff801bc-5b61-45e5-8d3f-97d025048d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:51.620186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff801bc-5b61-45e5-8d3f-97d025048d69 None admin] PATH_INFO: `/v3/users/84ee2bd3bcdc4950a4fd029642d833a3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:51.621460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff801bc-5b61-45e5-8d3f-97d025048d69 None admin] RBAC: Authorizing `identity:get_user(user_id=84ee2bd3bcdc4950a4fd029642d833a3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:51.623006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff801bc-5b61-45e5-8d3f-97d025048d69 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:51.625854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 128/266] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:51 2022] GET /identity/v3/users/84ee2bd3bcdc4950a4fd029642d833a3 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:51.633700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:51.761497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:51.806059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47200797-fce3-459b-8d58-4b081d8477b3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:51.850653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0729cd-06c5-44aa-8084-b313bbfebe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:51.851649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0729cd-06c5-44aa-8084-b313bbfebe96 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:51.852134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0729cd-06c5-44aa-8084-b313bbfebe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:51.852643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0729cd-06c5-44aa-8084-b313bbfebe96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:51.940299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0729cd-06c5-44aa-8084-b313bbfebe96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:51.942254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0729cd-06c5-44aa-8084-b313bbfebe96 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:52.040136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:52.087723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b66c26-13ee-4fc6-a3dc-aeb6cf382929 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.088042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b66c26-13ee-4fc6-a3dc-aeb6cf382929 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.088297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b66c26-13ee-4fc6-a3dc-aeb6cf382929 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.089697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 139/267] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:52.092441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 129/268] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:52.095008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b66c26-13ee-4fc6-a3dc-aeb6cf382929 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:52.095405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe7aa2-803d-4ce4-bd18-91083863561a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.095885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe7aa2-803d-4ce4-bd18-91083863561a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.096355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe7aa2-803d-4ce4-bd18-91083863561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.127961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8fbe7aa2-803d-4ce4-bd18-91083863561a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:52.129200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8fbe7aa2-803d-4ce4-bd18-91083863561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:52.254953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 140/269] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:52.274993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b66c26-13ee-4fc6-a3dc-aeb6cf382929 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:52.279935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe7aa2-803d-4ce4-bd18-91083863561a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:52.356445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:52.357129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.357416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.357693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.359942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:52.361422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:52.363257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 130/270] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:52 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:52.384474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:52.434467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:52.441728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe7aa2-803d-4ce4-bd18-91083863561a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:52.475837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:52.482442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:52.483395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.483884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.484366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.507202 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:52.512099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:52.513023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:52.513826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:52.522823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 141/271] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:52.523808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:52.603320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849cc89-1368-4434-92c3-a73e79ac1473 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:52.639698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510cb2b-1511-4332-8022-3226ead62d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:52.640320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510cb2b-1511-4332-8022-3226ead62d49 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.640634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510cb2b-1511-4332-8022-3226ead62d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.640912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510cb2b-1511-4332-8022-3226ead62d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.641824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510cb2b-1511-4332-8022-3226ead62d49 None admin] RBAC: Adding query filter params (role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=84ee2bd3bcdc4950a4fd029642d833a3) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:52.642140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510cb2b-1511-4332-8022-3226ead62d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:52.643565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510cb2b-1511-4332-8022-3226ead62d49 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:52.650624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 131/272] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:52 2022] GET /identity/v3/role_assignments?user.id=84ee2bd3bcdc4950a4fd029642d833a3&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:52.651762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198df66c-c074-4361-bd41-4f9eea9cdd5b None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.652048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198df66c-c074-4361-bd41-4f9eea9cdd5b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.652323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198df66c-c074-4361-bd41-4f9eea9cdd5b None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.658247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 132/273] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:52.659825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a08f8-e06e-436a-afc0-9a4861efe084 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.660326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a08f8-e06e-436a-afc0-9a4861efe084 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.660852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a08f8-e06e-436a-afc0-9a4861efe084 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.667972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 133/274] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:52.670149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ecb6-6015-42fe-aab8-2c7a76706007 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.670997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ecb6-6015-42fe-aab8-2c7a76706007 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.671533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ecb6-6015-42fe-aab8-2c7a76706007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.682393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe882-c1f5-4d37-8545-7a3bd2a3a5e2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:52.698717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5302ecb6-6015-42fe-aab8-2c7a76706007 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:52.701268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5302ecb6-6015-42fe-aab8-2c7a76706007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:52.738235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482f404-0c2e-489c-aa2b-69399cc07163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:52.740052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482f404-0c2e-489c-aa2b-69399cc07163 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.740567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482f404-0c2e-489c-aa2b-69399cc07163 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.741348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482f404-0c2e-489c-aa2b-69399cc07163 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.742826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482f404-0c2e-489c-aa2b-69399cc07163 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:52.743782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482f404-0c2e-489c-aa2b-69399cc07163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:52.745990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482f404-0c2e-489c-aa2b-69399cc07163 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:52.763704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 142/275] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:52.763704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04723-1570-4089-8e11-caaf40a076e2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.763704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04723-1570-4089-8e11-caaf40a076e2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.763704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04723-1570-4089-8e11-caaf40a076e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.787663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9cc04723-1570-4089-8e11-caaf40a076e2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:52.790096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9cc04723-1570-4089-8e11-caaf40a076e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:52.796815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 134/276] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:52.802817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302ecb6-6015-42fe-aab8-2c7a76706007 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:52.809285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:52.837629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:52.882821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 143/277] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:52.886267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04723-1570-4089-8e11-caaf40a076e2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:52.925831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302ecb6-6015-42fe-aab8-2c7a76706007 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:52.971067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:52.972032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:52.972583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:52.973087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] PATH_INFO: `/v3/users/7c99098c4aac468ab030613c03e7ed18` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:52.974891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=7c99098c4aac468ab030613c03e7ed18)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:52.976706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:52.978588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 135/278] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:52 2022] GET /identity/v3/users/7c99098c4aac468ab030613c03e7ed18 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:52.983847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:52.992913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04723-1570-4089-8e11-caaf40a076e2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.028585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.029171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.029458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.029741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.030897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:53.032360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:53.034018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 144/279] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:53.037424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:53.057315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:53.093878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:53.102240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15720dd-49e2-4488-a41f-962f46d4f6a7 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.138124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.139081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.139370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.139651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.145399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04682b0-2a05-4d8e-a2b3-9a9d9db5f6af None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.151304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:53.157600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:53.164098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 136/280] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:53.165177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:53.191119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.191804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.192092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.192369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.194052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:53.195554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:53.197272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 145/281] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:53 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:53.202588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:53.275048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdf4ae-a5b6-4c25-8292-9690ed75a36e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.298298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:53.312211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.312955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.313247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.313531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.314229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:53.315687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:53.315923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c908-1062-44bc-b023-9eb9c03e111b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.340442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 137/282] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:12:53 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:53.353601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:53.358960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.359993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.360483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.360984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.361921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:53.362437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:53.364165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:53.372003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:53.375950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 146/283] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:53.384942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:53.499167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fdfdd-4df0-4ab4-986d-312e4de33323 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.540656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7dc1b-7639-4964-a060-f8530abc0469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.541347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7dc1b-7639-4964-a060-f8530abc0469 None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.541633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7dc1b-7639-4964-a060-f8530abc0469 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.542571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc62bd-0c27-4a2e-ae28-108584072d42 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.543007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7dc1b-7639-4964-a060-f8530abc0469 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/7c99098c4aac468ab030613c03e7ed18/roles/f051e915d2d1479ba724a09298df84b7` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.544836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7dc1b-7639-4964-a060-f8530abc0469 None admin] RBAC: Authorizing `identity:create_grant(project_id=b518d31d96ad4d21b04a249253c6a4d5, user_id=7c99098c4aac468ab030613c03e7ed18, role_id=f051e915d2d1479ba724a09298df84b7)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:53.546310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7dc1b-7639-4964-a060-f8530abc0469 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:53.561199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 138/284] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:12:53 2022] PUT /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/users/7c99098c4aac468ab030613c03e7ed18/roles/f051e915d2d1479ba724a09298df84b7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:53.562359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ff80c-008d-4233-bb79-2b70f32d20ff None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.562649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ff80c-008d-4233-bb79-2b70f32d20ff None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.562974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ff80c-008d-4233-bb79-2b70f32d20ff None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.564284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 139/285] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:53.570540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf7d6b-6d19-497e-ada8-7f74af981333 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.570877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf7d6b-6d19-497e-ada8-7f74af981333 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.571058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf7d6b-6d19-497e-ada8-7f74af981333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.594552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-24cf7d6b-6d19-497e-ada8-7f74af981333 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:53.595778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-24cf7d6b-6d19-497e-ada8-7f74af981333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:53.596121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.596838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.597753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.597753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] PATH_INFO: `/v3/users/c5174a3d104040e18179bed57a420da9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.614628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=c5174a3d104040e18179bed57a420da9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:53.616299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:53.618159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 147/286] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:53 2022] GET /identity/v3/users/c5174a3d104040e18179bed57a420da9 => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:53.624318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:53.635759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:53.693904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:53.696916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 140/287] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:53.719682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf7d6b-6d19-497e-ada8-7f74af981333 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:53.780419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b90c-b96c-4653-a851-c034ce3ee9a9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.820598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.821184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.821469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.821745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.822535 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, user.id=c5174a3d104040e18179bed57a420da9, scope.system=all) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:53.822849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:53.824304 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:53.830290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 148/288] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 29 06:12:53 2022] GET /identity/v3/role_assignments?user.id=c5174a3d104040e18179bed57a420da9&scope.system=all&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:53.847953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf7d6b-6d19-497e-ada8-7f74af981333 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:53.888199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:53.889210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:53.889680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:53.890095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:53.902013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:53.907586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:53.908715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:53.909689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:53.911818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 141/289] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:12:53.917864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:54.041581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:54.081729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9836-1b5a-4b69-8d47-248a32de5857 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:54.135853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:54.136486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:54.136802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:54.137083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:54.137808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:54.138112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:54.139505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:54.150376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 149/290] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:54.157344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:54.262094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:54.310886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd924d2-55b3-4444-9c93-ab2d8f06347c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:54.351040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:54.351968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:54.352392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:54.352829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] PATH_INFO: `/v3/users/b37c9fb8dd9c4b21a6dfcc0ff42062da` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:54.354156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] RBAC: Authorizing `identity:get_user(user_id=b37c9fb8dd9c4b21a6dfcc0ff42062da)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:54.355761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:54.357842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 142/291] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:54 2022] GET /identity/v3/users/b37c9fb8dd9c4b21a6dfcc0ff42062da => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:54.362999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:54.516806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238ae73-44e0-409d-9816-549d5408f57b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:54.548147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:54.555329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:54.555967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:54.556261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:54.556562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:54.649271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:54.651369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:54.767800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 150/292] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:54.774306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:54.913152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f05ca-cef5-4a46-b146-d1438c28cd32 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:54.939552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:54.952614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0949231-4283-4bab-b7a8-3fa7e0003737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:54.953273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949231-4283-4bab-b7a8-3fa7e0003737 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:54.953558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949231-4283-4bab-b7a8-3fa7e0003737 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:54.953834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949231-4283-4bab-b7a8-3fa7e0003737 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:54.954994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949231-4283-4bab-b7a8-3fa7e0003737 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:54.956734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949231-4283-4bab-b7a8-3fa7e0003737 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:54.959362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 143/293] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:54 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:54.965754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:55.024321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:55.113214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632677-9cab-4f38-ba4e-4d5be1e9ba60 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:55.167402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49531c24-7896-4d3d-888d-bac15ba12dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:55.168394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49531c24-7896-4d3d-888d-bac15ba12dfe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:55.168911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49531c24-7896-4d3d-888d-bac15ba12dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:55.169409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49531c24-7896-4d3d-888d-bac15ba12dfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:55.170395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49531c24-7896-4d3d-888d-bac15ba12dfe None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=b37c9fb8dd9c4b21a6dfcc0ff42062da) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:55.170927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49531c24-7896-4d3d-888d-bac15ba12dfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:55.172585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49531c24-7896-4d3d-888d-bac15ba12dfe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:55.185990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 151/294] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:54 2022] GET /identity/v3/role_assignments?user.id=b37c9fb8dd9c4b21a6dfcc0ff42062da&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:55.912231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899dd7e-6ce9-473e-904e-824dce85ef75 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:55.913270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899dd7e-6ce9-473e-904e-824dce85ef75 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:55.913760 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899dd7e-6ce9-473e-904e-824dce85ef75 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:55.915549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 144/295] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:55.920959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0060c4c-29be-425e-a48a-7b4d747831a9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:55.921789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0060c4c-29be-425e-a48a-7b4d747831a9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:55.922261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0060c4c-29be-425e-a48a-7b4d747831a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:55.941235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a0060c4c-29be-425e-a48a-7b4d747831a9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:55.943276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a0060c4c-29be-425e-a48a-7b4d747831a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:56.019931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 152/296] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:56.041648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899dd7e-6ce9-473e-904e-824dce85ef75 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.075746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226ee13-bac2-4757-ab8d-1dfe56520acb None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.076139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226ee13-bac2-4757-ab8d-1dfe56520acb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.076307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226ee13-bac2-4757-ab8d-1dfe56520acb None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.078215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 153/297] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:56.083357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d914-9843-41a9-9fe9-a867e0e3396b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.083643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d914-9843-41a9-9fe9-a867e0e3396b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.083923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d914-9843-41a9-9fe9-a867e0e3396b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.102002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-feb0d914-9843-41a9-9fe9-a867e0e3396b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:56.103172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-feb0d914-9843-41a9-9fe9-a867e0e3396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:56.125854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:56.164166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899dd7e-6ce9-473e-904e-824dce85ef75 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.175229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 154/298] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:56.176311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d87f4-46ca-45f2-91ca-339471a1bf99 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.176666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d87f4-46ca-45f2-91ca-339471a1bf99 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.177478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d87f4-46ca-45f2-91ca-339471a1bf99 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.178732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 155/299] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:56.183902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e4bf-e09d-4152-94a7-587c529f8d46 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.184178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e4bf-e09d-4152-94a7-587c529f8d46 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.184460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e4bf-e09d-4152-94a7-587c529f8d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.203170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6747e4bf-e09d-4152-94a7-587c529f8d46 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:56.204345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6747e4bf-e09d-4152-94a7-587c529f8d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:56.205029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.206169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.206645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.206997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.208146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:56.209940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:56.211692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 145/300] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:56.212376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.275154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:56.277270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 156/301] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:56.281421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e4bf-e09d-4152-94a7-587c529f8d46 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.314441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a465b-f99e-4455-a6ac-ff53a37b2c38 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.347490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d822125-f9eb-43e8-aa5f-fbfb56bcf1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.348135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822125-f9eb-43e8-aa5f-fbfb56bcf1bd None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.348418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822125-f9eb-43e8-aa5f-fbfb56bcf1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.348722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822125-f9eb-43e8-aa5f-fbfb56bcf1bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.359223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d822125-f9eb-43e8-aa5f-fbfb56bcf1bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-0d822125-f9eb-43e8-aa5f-fbfb56bcf1bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:56.365219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:56.366237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:56.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 146/302] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:56.368815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ce210-e442-40b7-ac0f-907ceebcd1b0 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.369144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ce210-e442-40b7-ac0f-907ceebcd1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.369424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ce210-e442-40b7-ac0f-907ceebcd1b0 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.369740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:56.370814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 147/303] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:56.373297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ce210-e442-40b7-ac0f-907ceebcd1b0 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.385365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e4bf-e09d-4152-94a7-587c529f8d46 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.418774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.419387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.419664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.419935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] PATH_INFO: `/v3/groups` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.420650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:56.422074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:56.437432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 157/304] 10.222.0.33 () {60 vars in 1134 bytes} [Fri Jul 29 06:12:56 2022] POST /identity/v3/groups => generated 229 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:56.441547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.487546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ce210-e442-40b7-ac0f-907ceebcd1b0 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.517413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:56.530543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4660f2c-a8c7-4467-a2b1-836e72597dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.531120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4660f2c-a8c7-4467-a2b1-836e72597dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.531526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4660f2c-a8c7-4467-a2b1-836e72597dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.531804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4660f2c-a8c7-4467-a2b1-836e72597dc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.545294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4660f2c-a8c7-4467-a2b1-836e72597dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-e4660f2c-a8c7-4467-a2b1-836e72597dc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:56.550874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:56.552654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:56.554304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 148/305] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:56.555252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188ca30-1fdc-47d7-bd34-417fa2fabdc3 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.555252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188ca30-1fdc-47d7-bd34-417fa2fabdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.555252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188ca30-1fdc-47d7-bd34-417fa2fabdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.560814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f19a0-22d2-4dd2-a009-6b60397bb165 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.575537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2188ca30-1fdc-47d7-bd34-417fa2fabdc3 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:56.576943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2188ca30-1fdc-47d7-bd34-417fa2fabdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:56.600303 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.600968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.601258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.601516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.602702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:56.603293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:56.604948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:56.615257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 158/306] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:56.616071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.628091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:56.654761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 149/307] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:56.655824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188ca30-1fdc-47d7-bd34-417fa2fabdc3 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.729685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004a57a-9a86-48d5-be02-0e4d757ef5cc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.766544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.767171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.767457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.767736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.768472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:56.768810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:56.769534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188ca30-1fdc-47d7-bd34-417fa2fabdc3 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.770187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:56.779515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 159/308] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:56.780345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.791315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:56.818386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.819045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.819324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.819597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] PATH_INFO: `/v3/users/7e561fce21cc43a88b7cb1daaf626be0` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.837526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=7e561fce21cc43a88b7cb1daaf626be0)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:56.839601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:56.842588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 150/309] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/users/7e561fce21cc43a88b7cb1daaf626be0 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:56.843228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.855854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:56.889417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190e66c-71a6-4e06-bec9-76d0203a3099 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.926740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:56.927410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:56.927680 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:56.927949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:56.938486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:56.943997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:56.945112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:56.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 160/310] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:56.946586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:56.962469 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0bbe3-826e-4cde-8e54-525a1fe28ed0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:56.999520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:57.000169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:57.000449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:57.000772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] PATH_INFO: `/v3/users/c5174a3d104040e18179bed57a420da9` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:57.001939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] RBAC: Authorizing `identity:get_user(user_id=c5174a3d104040e18179bed57a420da9)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:57.003363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:57.004985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 151/311] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/users/c5174a3d104040e18179bed57a420da9 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:57.005761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:57.053940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:57.060703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb7031-6b60-442e-b573-f6c26b44d2c2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:57.091825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:57.098070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:57.098697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:57.099373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:57.099767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:57.100562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] RBAC: Adding query filter params (role.id=f051e915d2d1479ba724a09298df84b7, user.id=7e561fce21cc43a88b7cb1daaf626be0, scope.system=all) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:57.100891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:57.102393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:57.116581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 161/312] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 29 06:12:56 2022] GET /identity/v3/role_assignments?user.id=7e561fce21cc43a88b7cb1daaf626be0&scope.system=all&role.id=f051e915d2d1479ba724a09298df84b7 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:57.116581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:57.129948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e76e26-9306-4be1-8ce3-822a4d005f39 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:57.186700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:57.187348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:57.187890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:57.189430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:57.190197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:57.190516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:57.191969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:57.204393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 152/313] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:57.213700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:57.228579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240825-181d-4f75-920e-23b98e394c38 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:57.272290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:57.273377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:57.273891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:57.274398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] PATH_INFO: `/v3/system/users/c5174a3d104040e18179bed57a420da9/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:57.276462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c5174a3d104040e18179bed57a420da9, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:57.278266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:57.289393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 162/314] 10.222.0.33 () {58 vars in 1334 bytes} [Fri Jul 29 06:12:57 2022] PUT /identity/v3/system/users/c5174a3d104040e18179bed57a420da9/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:12:57.340890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ae3d2-147e-4461-89c8-b339c23fa726 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:57.343100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:57.385936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:57.386592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:57.386874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:57.387146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] PATH_INFO: `/v3/users/7c99098c4aac468ab030613c03e7ed18` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:57.388346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] RBAC: Authorizing `identity:get_user(user_id=7c99098c4aac468ab030613c03e7ed18)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:57.389881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:57.391765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 153/315] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:57 2022] GET /identity/v3/users/7c99098c4aac468ab030613c03e7ed18 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:57.398324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:57.464661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:57.511435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb6c92-9563-496f-a3a6-786a358b1fb7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:57.547242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:57.547867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:57.548200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:57.548601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:57.626115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:57.627563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:57.686413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:57.733445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 163/316] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:12:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:12:57.740403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:57.848881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acb10a-f7d5-4b89-b3d6-709a85d559d8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:57.884165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1325edc-a706-48a4-88fc-d92c8914ba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:57.885114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1325edc-a706-48a4-88fc-d92c8914ba5b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:57.885570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1325edc-a706-48a4-88fc-d92c8914ba5b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:57.886040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1325edc-a706-48a4-88fc-d92c8914ba5b None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:57.887428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1325edc-a706-48a4-88fc-d92c8914ba5b None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:57.889065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1325edc-a706-48a4-88fc-d92c8914ba5b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:57.891241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 154/317] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:12:57 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:57.897818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:58.002536 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7dbc-aec9-4608-b581-498894265516 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:58.039663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee39447-7ca2-4cb9-8455-f877bd8c81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:58.040292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39447-7ca2-4cb9-8455-f877bd8c81de None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:58.040608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39447-7ca2-4cb9-8455-f877bd8c81de None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:58.040895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39447-7ca2-4cb9-8455-f877bd8c81de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:58.041682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee39447-7ca2-4cb9-8455-f877bd8c81de None admin] RBAC: Adding query filter params (role.id=f051e915d2d1479ba724a09298df84b7, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5, user.id=7c99098c4aac468ab030613c03e7ed18) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:58.041996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee39447-7ca2-4cb9-8455-f877bd8c81de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:58.043431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee39447-7ca2-4cb9-8455-f877bd8c81de None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:58.049667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 164/318] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:12:57 2022] GET /identity/v3/role_assignments?user.id=7c99098c4aac468ab030613c03e7ed18&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=f051e915d2d1479ba724a09298df84b7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:58.534544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a73d5d-26eb-4019-af73-10f9dc49786b None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:58.535085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a73d5d-26eb-4019-af73-10f9dc49786b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:58.535085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a73d5d-26eb-4019-af73-10f9dc49786b None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:58.537081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 155/319] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:58.542211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6414cd0-8bcf-45a5-98ef-8ff66f810d21 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:58.542512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6414cd0-8bcf-45a5-98ef-8ff66f810d21 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:58.542796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6414cd0-8bcf-45a5-98ef-8ff66f810d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:58.560849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c6414cd0-8bcf-45a5-98ef-8ff66f810d21 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:58.562039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c6414cd0-8bcf-45a5-98ef-8ff66f810d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:58.579922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:58.632586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 165/320] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:58.654689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a73d5d-26eb-4019-af73-10f9dc49786b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:58.696103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:58.761778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a73d5d-26eb-4019-af73-10f9dc49786b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:58.795274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c667bab-ce5a-4286-8244-729b63f7af84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:58.795837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c667bab-ce5a-4286-8244-729b63f7af84 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:58.796117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c667bab-ce5a-4286-8244-729b63f7af84 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:58.796385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c667bab-ce5a-4286-8244-729b63f7af84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:58.797530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c667bab-ce5a-4286-8244-729b63f7af84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:58.798939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c667bab-ce5a-4286-8244-729b63f7af84 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:58.800703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 156/321] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:12:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:58.806119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6414cd0-8bcf-45a5-98ef-8ff66f810d21 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:58.905496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6414cd0-8bcf-45a5-98ef-8ff66f810d21 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:58.938926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288f0a9-b77a-4b51-bba8-fc3a21c82931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:58.939547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f0a9-b77a-4b51-bba8-fc3a21c82931 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:58.939826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f0a9-b77a-4b51-bba8-fc3a21c82931 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:58.940100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f0a9-b77a-4b51-bba8-fc3a21c82931 None admin] PATH_INFO: `/v3/groups` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:58.940785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288f0a9-b77a-4b51-bba8-fc3a21c82931 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:58.942312 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288f0a9-b77a-4b51-bba8-fc3a21c82931 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:58.959679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 166/322] 10.222.0.33 () {60 vars in 1134 bytes} [Fri Jul 29 06:12:58 2022] POST /identity/v3/groups => generated 226 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:12:59.213914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b321b6-7c10-4126-9fe2-8eb2036dbcd5 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.214301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b321b6-7c10-4126-9fe2-8eb2036dbcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.214491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b321b6-7c10-4126-9fe2-8eb2036dbcd5 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.215984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 157/323] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:12:59.221197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecf8d2-486e-4a48-985b-ae1e446b67ae None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.221660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecf8d2-486e-4a48-985b-ae1e446b67ae None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.221744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecf8d2-486e-4a48-985b-ae1e446b67ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.240671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-79ecf8d2-486e-4a48-985b-ae1e446b67ae None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:59.241849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-79ecf8d2-486e-4a48-985b-ae1e446b67ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:59.250022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:59.321488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 167/324] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:12:59.343359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b321b6-7c10-4126-9fe2-8eb2036dbcd5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:59.450673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b321b6-7c10-4126-9fe2-8eb2036dbcd5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:59.458626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:59.466504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b663ea-9b64-4008-b2d6-a0a581fce2e4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.466759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b663ea-9b64-4008-b2d6-a0a581fce2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.467006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b663ea-9b64-4008-b2d6-a0a581fce2e4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.468665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 168/325] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:12:59.474209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a21bc5-8972-4ca9-9070-c6b7af20e95e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.474739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a21bc5-8972-4ca9-9070-c6b7af20e95e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.475533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a21bc5-8972-4ca9-9070-c6b7af20e95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.493270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:59.494003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.494287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.494594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.499915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-31a21bc5-8972-4ca9-9070-c6b7af20e95e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:12:59.501130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-31a21bc5-8972-4ca9-9070-c6b7af20e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:12:59.507010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:59.512168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:59.513046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:12:59.513822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 158/326] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:59.519669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:59.597837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 169/327] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:12:59.634573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a21bc5-8972-4ca9-9070-c6b7af20e95e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:59.647841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c8a7c-6821-4719-9dda-3f826307840a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:59.684823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:59.685456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.685737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.686012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.686725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:12:59.687028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:59.688403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:59.697894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 159/328] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:59.705715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:59.755052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:59.762134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:12:59.804788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a21bc5-8972-4ca9-9070-c6b7af20e95e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:59.846857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:59.847415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e5b7-6cc1-4684-b59b-2d1691c56c3b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:12:59.847525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.847776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.848052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.871918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:12:59.877282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:12:59.878152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:12:59.878956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:12:59.883356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 170/329] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:12:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:12:59.884718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:12:59.889546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:12:59.891654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:12:59.892181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:12:59.892871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] PATH_INFO: `/v3/users/7e561fce21cc43a88b7cb1daaf626be0` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:12:59.895398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] RBAC: Authorizing `identity:get_user(user_id=7e561fce21cc43a88b7cb1daaf626be0)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:12:59.897871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:12:59.903304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 160/330] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:12:59 2022] GET /identity/v3/users/7e561fce21cc43a88b7cb1daaf626be0 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:12:59.924532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:00.079518 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673d9d4-ee5f-4518-945f-8173ad82a7c4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:00.119183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:00.123198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:00.128509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:00.130890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:00.131373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:00.131841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:00.132799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:00.133779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:00.135548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:00.138964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3ddf8-bc5a-4611-9f0a-eb5769c07a8b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:00.147193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 171/331] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:12:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:00.155412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:00.202299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:00.202979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:00.203259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:00.203538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] PATH_INFO: `/v3/system/users/7e561fce21cc43a88b7cb1daaf626be0/roles/f051e915d2d1479ba724a09298df84b7` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:00.205078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e561fce21cc43a88b7cb1daaf626be0, role_id=f051e915d2d1479ba724a09298df84b7)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:00.206708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:00.225704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 161/332] 10.222.0.33 () {58 vars in 1334 bytes} [Fri Jul 29 06:12:59 2022] PUT /identity/v3/system/users/7e561fce21cc43a88b7cb1daaf626be0/roles/f051e915d2d1479ba724a09298df84b7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:13:00.313487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31069b-77db-469c-b628-7a33891f0116 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:00.360056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da039c1-317e-4882-a3f0-7a0009115196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:00.360741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da039c1-317e-4882-a3f0-7a0009115196 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:00.361151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da039c1-317e-4882-a3f0-7a0009115196 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:00.361417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da039c1-317e-4882-a3f0-7a0009115196 None admin] PATH_INFO: `/v3/users/c5174a3d104040e18179bed57a420da9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:00.362859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da039c1-317e-4882-a3f0-7a0009115196 None admin] RBAC: Authorizing `identity:get_user(user_id=c5174a3d104040e18179bed57a420da9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:00.364342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da039c1-317e-4882-a3f0-7a0009115196 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:00.366364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 172/333] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:13:00 2022] GET /identity/v3/users/c5174a3d104040e18179bed57a420da9 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:00.372466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:00.482479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70037e-feba-48c6-b818-5d887f5f31bd None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:00.519672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710473-6ec7-4b1f-a259-e5d4d8b38837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:00.520311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710473-6ec7-4b1f-a259-e5d4d8b38837 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:00.520705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710473-6ec7-4b1f-a259-e5d4d8b38837 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:00.521093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710473-6ec7-4b1f-a259-e5d4d8b38837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:00.522142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710473-6ec7-4b1f-a259-e5d4d8b38837 None admin] RBAC: Adding query filter params (role.id=5f97bcaf49fe487d8def3bf0991dcf15, user.id=c5174a3d104040e18179bed57a420da9, scope.system=all) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:00.522515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710473-6ec7-4b1f-a259-e5d4d8b38837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:00.524007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710473-6ec7-4b1f-a259-e5d4d8b38837 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:00.539388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 162/334] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 29 06:13:00 2022] GET /identity/v3/role_assignments?user.id=c5174a3d104040e18179bed57a420da9&scope.system=all&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:01.475633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcd3a2-fff2-4cd5-940c-2716274cec10 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:01.476174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcd3a2-fff2-4cd5-940c-2716274cec10 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:01.476174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcd3a2-fff2-4cd5-940c-2716274cec10 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:01.477841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 173/335] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:01.483150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838417b5-049c-452d-8bef-37be8602d0c7 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:01.483550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838417b5-049c-452d-8bef-37be8602d0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:01.483715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838417b5-049c-452d-8bef-37be8602d0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:01.503711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-838417b5-049c-452d-8bef-37be8602d0c7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:01.505544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-838417b5-049c-452d-8bef-37be8602d0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:01.519732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:01.580696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 163/336] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:01.602078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcd3a2-fff2-4cd5-940c-2716274cec10 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:01.675923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:01.733341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcd3a2-fff2-4cd5-940c-2716274cec10 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:01.782339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:01.782989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:01.783271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:01.783543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:01.795483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:01.800933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:01.801830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:01.802615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:01.806115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 174/337] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:01.808476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838417b5-049c-452d-8bef-37be8602d0c7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:01.923550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838417b5-049c-452d-8bef-37be8602d0c7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:01.967546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78adbd-37a3-4daa-8620-3a1f5a691d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:01.968179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78adbd-37a3-4daa-8620-3a1f5a691d8f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:01.968462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78adbd-37a3-4daa-8620-3a1f5a691d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:01.968762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78adbd-37a3-4daa-8620-3a1f5a691d8f None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:01.969481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78adbd-37a3-4daa-8620-3a1f5a691d8f None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:01.969785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78adbd-37a3-4daa-8620-3a1f5a691d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:01.971207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78adbd-37a3-4daa-8620-3a1f5a691d8f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:01.980218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 164/338] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:13:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:01.985819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:02.076618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427831d0-65cc-4115-8f71-0b404be5c191 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.076924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427831d0-65cc-4115-8f71-0b404be5c191 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.077199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427831d0-65cc-4115-8f71-0b404be5c191 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.078813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 165/339] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:02.084065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451390d-047f-47c7-9b2d-ef07c7d30ea1 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.084351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451390d-047f-47c7-9b2d-ef07c7d30ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.084699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451390d-047f-47c7-9b2d-ef07c7d30ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.091878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bfb8f-05f4-4d01-9b15-4570dbec984f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:02.120954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:02.127630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c451390d-047f-47c7-9b2d-ef07c7d30ea1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:02.129791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c451390d-047f-47c7-9b2d-ef07c7d30ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:02.143564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:02.144193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:02.144301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.144464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.144820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.281827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:02.283220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:02.288104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 166/340] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:02.321632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451390d-047f-47c7-9b2d-ef07c7d30ea1 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:02.421052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 175/341] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:02.423271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:02.454157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:02.556147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46550e19-8247-4f56-9bf1-f1b62129924b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:02.600116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76b8d-3962-4e53-b6b2-43244068a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:02.600810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76b8d-3962-4e53-b6b2-43244068a932 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.601094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76b8d-3962-4e53-b6b2-43244068a932 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.601371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76b8d-3962-4e53-b6b2-43244068a932 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.602546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76b8d-3962-4e53-b6b2-43244068a932 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:02.604000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76b8d-3962-4e53-b6b2-43244068a932 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:02.605686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 176/342] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:02 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:02.609720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451390d-047f-47c7-9b2d-ef07c7d30ea1 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:02.611150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef150d6-5204-44e1-ba27-a0bf5cb08824 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.611438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef150d6-5204-44e1-ba27-a0bf5cb08824 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.611718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef150d6-5204-44e1-ba27-a0bf5cb08824 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.613615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 177/343] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:02.617820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef150d6-5204-44e1-ba27-a0bf5cb08824 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:02.655923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281e968-29c1-4a9e-a0fc-107d013ceb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:02.656860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281e968-29c1-4a9e-a0fc-107d013ceb1d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.657911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281e968-29c1-4a9e-a0fc-107d013ceb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.658332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281e968-29c1-4a9e-a0fc-107d013ceb1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.674350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:02.679330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281e968-29c1-4a9e-a0fc-107d013ceb1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-f281e968-29c1-4a9e-a0fc-107d013ceb1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:02.689002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:02.689898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:02.690676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 167/344] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:02.694415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a550-bee9-4116-845c-33f12e89f444 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.694707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a550-bee9-4116-845c-33f12e89f444 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.694989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a550-bee9-4116-845c-33f12e89f444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.729447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e802a550-bee9-4116-845c-33f12e89f444 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:02.730659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e802a550-bee9-4116-845c-33f12e89f444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:02.791961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:02.816943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef150d6-5204-44e1-ba27-a0bf5cb08824 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:02.867806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 168/345] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:02.868708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802a550-bee9-4116-845c-33f12e89f444 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:02.897777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:02.898445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:02.898725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:02.898996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:02.909192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:02.910651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:02.918063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 178/346] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:02 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:02.918949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:02.983940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802a550-bee9-4116-845c-33f12e89f444 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:03.021698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:03.031097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:03.031762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.032094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.032371 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.033402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:03.033712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:03.035099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:03.055679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 169/347] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:03.056783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:03.094454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e571ba-65d3-487c-b7be-073fb26dd269 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:03.159767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:03.160902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.161427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.161883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.187767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:03.194560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:03.195523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:03.196328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:03.202504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 179/348] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:03.203560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:03.226511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0462c9e-2e0d-4669-9314-e080fb7cea25 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:03.238660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:03.274880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:03.275518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.275921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.276210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.277082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:03.277408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:03.278838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:03.285886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 170/349] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:03 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:03.290276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:03.401025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:03.478657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea95309-a958-45bd-8042-3f7425bc402b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:03.500735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a3f94-6f9e-4a91-9cde-9f62efb6984b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:03.532843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2229-852d-4a51-a6c5-4b78c3938b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:03.533588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2229-852d-4a51-a6c5-4b78c3938b4a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.534137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2229-852d-4a51-a6c5-4b78c3938b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.534563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2229-852d-4a51-a6c5-4b78c3938b4a None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.535522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2229-852d-4a51-a6c5-4b78c3938b4a None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:03.535984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2229-852d-4a51-a6c5-4b78c3938b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:03.537947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2229-852d-4a51-a6c5-4b78c3938b4a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:03.547787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 171/350] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:13:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:03.549440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068bdb-ffb1-4c66-bda9-6e364254e898 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.554160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068bdb-ffb1-4c66-bda9-6e364254e898 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.554669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068bdb-ffb1-4c66-bda9-6e364254e898 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.556329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 172/351] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:03.557244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068bdb-ffb1-4c66-bda9-6e364254e898 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:03.560733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580b620-7cd4-4ec4-b0b1-6e9ed6273d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:03.564574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b620-7cd4-4ec4-b0b1-6e9ed6273d46 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.564574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b620-7cd4-4ec4-b0b1-6e9ed6273d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.564574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b620-7cd4-4ec4-b0b1-6e9ed6273d46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.666168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580b620-7cd4-4ec4-b0b1-6e9ed6273d46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:03.667585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580b620-7cd4-4ec4-b0b1-6e9ed6273d46 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:03.689100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:03.744464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068bdb-ffb1-4c66-bda9-6e364254e898 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:03.765946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:03.791409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a067f4f-98e9-4b3c-ad17-cce6ac4e5783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:03.792290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a067f4f-98e9-4b3c-ad17-cce6ac4e5783 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.792782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a067f4f-98e9-4b3c-ad17-cce6ac4e5783 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.793201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a067f4f-98e9-4b3c-ad17-cce6ac4e5783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.814750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 180/352] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:03.816336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8b07-a352-4b9b-b3d4-eea73d3925ec None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.816812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8b07-a352-4b9b-b3d4-eea73d3925ec None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.817263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8b07-a352-4b9b-b3d4-eea73d3925ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.839541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-caad8b07-a352-4b9b-b3d4-eea73d3925ec None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:03.841038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-caad8b07-a352-4b9b-b3d4-eea73d3925ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:03.886714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a067f4f-98e9-4b3c-ad17-cce6ac4e5783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:03.888219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a067f4f-98e9-4b3c-ad17-cce6ac4e5783 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:03.930010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 181/353] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:03.936564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f2f6a-4a87-4c25-a6ce-746dda30eb8d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:03.937217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f2f6a-4a87-4c25-a6ce-746dda30eb8d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:03.937716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f2f6a-4a87-4c25-a6ce-746dda30eb8d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:03.939496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 182/354] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:03.940340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f2f6a-4a87-4c25-a6ce-746dda30eb8d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.013055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:04.026287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 173/355] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:04.027026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247e2f-f59c-42ee-918a-b0aa2f456c9b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.027323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247e2f-f59c-42ee-918a-b0aa2f456c9b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.027605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247e2f-f59c-42ee-918a-b0aa2f456c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.052737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7e247e2f-f59c-42ee-918a-b0aa2f456c9b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:04.053944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7e247e2f-f59c-42ee-918a-b0aa2f456c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:04.074018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:04.091229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f2f6a-4a87-4c25-a6ce-746dda30eb8d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.137209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.138323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.138925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.139279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.140569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:04.142222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:04.144241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 183/356] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:03 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:04.144936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.151520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 174/357] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:04.152632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e247e2f-f59c-42ee-918a-b0aa2f456c9b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.277104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f422fbe-c530-49af-89bf-f54e2a8140f0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.293667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:04.300671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e247e2f-f59c-42ee-918a-b0aa2f456c9b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.323984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.324679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.324967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.325250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.341850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:04.346936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:04.347804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:04.348667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 184/358] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:04.349337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.355143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.355822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.356114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.356383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.360296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:04.362182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:04.365130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 175/359] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:04.365992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.388198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:04.491062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df496249-04f4-431c-aaf3-b0ef607d4b05 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.499381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2edf45-80b6-4f0b-95e4-d37f3a5be6ea None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.537771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffc7eb-8b53-41c9-afeb-46e3dfc00e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.538449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc7eb-8b53-41c9-afeb-46e3dfc00e88 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.538798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc7eb-8b53-41c9-afeb-46e3dfc00e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.539088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc7eb-8b53-41c9-afeb-46e3dfc00e88 None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.540244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffc7eb-8b53-41c9-afeb-46e3dfc00e88 None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:04.541844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffc7eb-8b53-41c9-afeb-46e3dfc00e88 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:04.543947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 185/360] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:04.544794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec1ae-0b3f-4d7b-9c4d-4bd7ac01c603 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.545083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec1ae-0b3f-4d7b-9c4d-4bd7ac01c603 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.545357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec1ae-0b3f-4d7b-9c4d-4bd7ac01c603 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.546005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.546716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 186/361] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:04.547276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.547796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.548335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.551453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ec1ae-0b3f-4d7b-9c4d-4bd7ac01c603 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.569688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:04.575251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:04.576139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:04.577052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 176/362] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:04.577706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.615167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:04.674769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:04.703631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ec1ae-0b3f-4d7b-9c4d-4bd7ac01c603 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.704432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff457e4-d0ef-4138-9253-1c03bbfc8136 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.749594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.750199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.750199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.750496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.752583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:04.753467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:04.755359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe4999-11a0-4e1f-bb40-fcaf35198f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.756574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 177/363] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:04.756704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe4999-11a0-4e1f-bb40-fcaf35198f2f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.756782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.756850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe4999-11a0-4e1f-bb40-fcaf35198f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.756850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe4999-11a0-4e1f-bb40-fcaf35198f2f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.757673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe4999-11a0-4e1f-bb40-fcaf35198f2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:04.758366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe4999-11a0-4e1f-bb40-fcaf35198f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:04.759303 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe4999-11a0-4e1f-bb40-fcaf35198f2f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:04.768164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 187/364] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:04.773588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdb190-218f-45df-865e-b99e2efaf5fb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.773873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdb190-218f-45df-865e-b99e2efaf5fb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.774202 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdb190-218f-45df-865e-b99e2efaf5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.798017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-99fdb190-218f-45df-865e-b99e2efaf5fb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:04.800106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-99fdb190-218f-45df-865e-b99e2efaf5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:04.885814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 188/365] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:04.892787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126688-4f90-485f-a8cf-6ab0013e454d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:04.892900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdb190-218f-45df-865e-b99e2efaf5fb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.929921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:04.939518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:04.939518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:04.939518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:04.939518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:04.939860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:04.939860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:04.940282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:04.948114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 178/366] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:04.948870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:04.963238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:05.031057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdb190-218f-45df-865e-b99e2efaf5fb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.079829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.080716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.081158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.081545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.082295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:05.082608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:05.084085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:05.093686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11fd72-7fd2-4206-8a0d-8486622fd15c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.094294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 189/367] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:05.097883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:05.141084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.141716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.141997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.142269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.144970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:05.144970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:05.145732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:05.153532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 179/368] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:04 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:05.154747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:05.233929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:05.242097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33343d27-90c5-4fcc-8808-a7443ccf6b72 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.287834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.289125 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.289653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.290137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] PATH_INFO: `/v3/users/7e561fce21cc43a88b7cb1daaf626be0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.293891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:05.295398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] RBAC: Authorizing `identity:get_user(user_id=7e561fce21cc43a88b7cb1daaf626be0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:05.297329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:05.300513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 190/369] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:13:05 2022] GET /identity/v3/users/7e561fce21cc43a88b7cb1daaf626be0 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:05.301782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:05.378975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90008c5b-ff14-48d0-be1d-7f449f169bd3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.431173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.432245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.432790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.433289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.437633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd6d34-186b-4d23-be3a-27c279c732f7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.447651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:05.453813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:05.454696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:05.455745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:05.458223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 180/370] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:05.459252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:05.476871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b6581-7264-4eda-8c9a-084de1ac46ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.477436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6581-7264-4eda-8c9a-084de1ac46ab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.477723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6581-7264-4eda-8c9a-084de1ac46ab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.479422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6581-7264-4eda-8c9a-084de1ac46ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.513789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:05.565866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b6581-7264-4eda-8c9a-084de1ac46ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:05.567378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b6581-7264-4eda-8c9a-084de1ac46ab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:05.584089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ddca6-1e20-4562-8c37-f566a01c94fa None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.618081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:05.623804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.624759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.625249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.627601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.628648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] RBAC: Adding query filter params (role.id=f051e915d2d1479ba724a09298df84b7, user.id=7e561fce21cc43a88b7cb1daaf626be0, scope.system=all) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:05.629239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:05.631513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:05.646586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 181/371] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 29 06:13:05 2022] GET /identity/v3/role_assignments?user.id=7e561fce21cc43a88b7cb1daaf626be0&scope.system=all&role.id=f051e915d2d1479ba724a09298df84b7 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:05.647853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:05.792490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aea53d-b8fa-4785-9fbf-8c5dce2e37ac None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.795949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 191/372] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:05.797378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650214de-c7aa-4812-b1e4-333079f8a505 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.797673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650214de-c7aa-4812-b1e4-333079f8a505 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.797950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650214de-c7aa-4812-b1e4-333079f8a505 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.799443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 192/373] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:05.805590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650214de-c7aa-4812-b1e4-333079f8a505 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:05.838844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e6243-23af-4d43-87c0-907310745f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.839576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e6243-23af-4d43-87c0-907310745f9c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.839868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e6243-23af-4d43-87c0-907310745f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.840147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e6243-23af-4d43-87c0-907310745f9c None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.841097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e6243-23af-4d43-87c0-907310745f9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:05.841414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e6243-23af-4d43-87c0-907310745f9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:05.842945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e6243-23af-4d43-87c0-907310745f9c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:05.853157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 182/374] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:05.854457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3353523-99cd-4a28-b2cd-19e31ae9f72f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.854751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3353523-99cd-4a28-b2cd-19e31ae9f72f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.855144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3353523-99cd-4a28-b2cd-19e31ae9f72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.878107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d3353523-99cd-4a28-b2cd-19e31ae9f72f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:05.879407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d3353523-99cd-4a28-b2cd-19e31ae9f72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:05.887238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:05.939279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:05.945522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650214de-c7aa-4812-b1e4-333079f8a505 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:05.964382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 183/375] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:05.965368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3353523-99cd-4a28-b2cd-19e31ae9f72f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:05.992040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:05.993002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:05.993283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:05.993562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:05.994858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:05.996301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:05.998384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 193/376] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:05 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:06.003494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:06.096491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3353523-99cd-4a28-b2cd-19e31ae9f72f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:06.130577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0a56b-8481-4424-a585-1febda211181 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:06.135559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:06.136312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:06.136718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:06.137070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:06.166825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:06.174806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:06.175730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:06.176018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:06.176295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:06.178782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:06.180637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:06.182573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 194/377] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:06.185399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:06.213089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:06.240528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:06.242135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:06.308461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8d49c-c48b-466f-9600-d758c8e8f357 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:06.352844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:06.353502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:06.353785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:06.354063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:06.366352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:06.372844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:06.373769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:06.374659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 195/378] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:06.375844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:06.376036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 184/379] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:06.382731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:06.456747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:06.488586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:06.505509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e75624-f2eb-4a48-a850-133f54a78fb7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:06.515632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa562b23-45ff-4d60-9301-3ecd22eb17fe None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:06.556850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:06.557492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:06.557778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:06.558062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:06.558862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:06.559177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:06.560687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:06.565397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:06.566041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:06.566382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:06.566911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:06.567026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 196/380] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:06.567969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:06.568367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:06.569930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:06.573619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:06.579515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 185/381] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:06.585801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:06.710275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d510d-ec21-444a-81b9-f16d3dfccfd6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:06.796248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a573-8294-412b-813d-7087cd3683cf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:06.814824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:06.815490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:06.815778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:06.816058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:06.818063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:06.819655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:06.821488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 186/382] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:06.829574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:06.845719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:06.850470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:06.870992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:06.871643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:06.871928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:06.872204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] PATH_INFO: `/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:06.875537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5535113d8e4690869b5d23e1b0a8b2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:06.877012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:06.881867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 197/383] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2 => generated 344 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:06.886725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:07.007281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceee544-74ef-423b-bdd4-f416709d6ccb None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:07.028972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a98712-0de9-4113-a3ce-57d5f611b701 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:07.063498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:07.064215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.064598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.064881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.068566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:07.069017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:07.070245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:07.071051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 187/384] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:07.071316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.071633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.071898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] PATH_INFO: `/v3/groups/d347c15848b44f508d72002ad220c5c2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.076198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:07.085233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] RBAC: Authorizing `identity:get_group(group_id=d347c15848b44f508d72002ad220c5c2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:07.086710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:07.092605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 198/385] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:06 2022] GET /identity/v3/groups/d347c15848b44f508d72002ad220c5c2 => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:07.112121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:07.179043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:07.179233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:07.200200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2f4e-ff9e-45cd-bb4b-6ead161d8504 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:07.231395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebd89-6a4c-4aad-bc5f-9c71a8629257 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:07.249639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c2933-14a8-449f-af4a-8d99ce29f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:07.250691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2933-14a8-449f-af4a-8d99ce29f840 None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.251214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2933-14a8-449f-af4a-8d99ce29f840 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.251685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2933-14a8-449f-af4a-8d99ce29f840 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/groups/50f1b0a57d224fba8b9501122ca52499/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.258094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c2933-14a8-449f-af4a-8d99ce29f840 None admin] RBAC: Authorizing `identity:create_grant(project_id=d90586d8e9094e65ad00907c4b6c8ddd, group_id=50f1b0a57d224fba8b9501122ca52499, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:07.259765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c2933-14a8-449f-af4a-8d99ce29f840 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:07.273065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27251550-5024-4647-bf30-1091a09a88e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:07.273712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27251550-5024-4647-bf30-1091a09a88e2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.274056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27251550-5024-4647-bf30-1091a09a88e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.274349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27251550-5024-4647-bf30-1091a09a88e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.276567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27251550-5024-4647-bf30-1091a09a88e2 None admin] RBAC: Adding query filter params (group.id=d347c15848b44f508d72002ad220c5c2, role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=cc5535113d8e4690869b5d23e1b0a8b2) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:07.276567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27251550-5024-4647-bf30-1091a09a88e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:07.277434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27251550-5024-4647-bf30-1091a09a88e2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:07.284802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 199/386] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:07 2022] GET /identity/v3/role_assignments?group.id=d347c15848b44f508d72002ad220c5c2&scope.project.id=cc5535113d8e4690869b5d23e1b0a8b2&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:07.287480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 188/387] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 29 06:13:07 2022] PUT /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/groups/50f1b0a57d224fba8b9501122ca52499/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:13:07.795633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1f11-0c8b-47ce-97f6-6032f0b31838 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.796306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1f11-0c8b-47ce-97f6-6032f0b31838 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.797262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1f11-0c8b-47ce-97f6-6032f0b31838 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.799166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 200/388] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:07.805252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec163f-7a9f-4d81-8d33-a1eab61b5589 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.805852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec163f-7a9f-4d81-8d33-a1eab61b5589 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.806355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec163f-7a9f-4d81-8d33-a1eab61b5589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.832677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-feec163f-7a9f-4d81-8d33-a1eab61b5589 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:07.834834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-feec163f-7a9f-4d81-8d33-a1eab61b5589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:07.858204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7d03-6526-4c9d-8069-cd6d81764e2d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.858849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7d03-6526-4c9d-8069-cd6d81764e2d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.859333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7d03-6526-4c9d-8069-cd6d81764e2d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.861201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 201/389] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:07.878390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197dd4-46ae-46f5-b265-3899d351a615 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:07.879016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197dd4-46ae-46f5-b265-3899d351a615 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:07.879542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197dd4-46ae-46f5-b265-3899d351a615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:07.901562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a5197dd4-46ae-46f5-b265-3899d351a615 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:07.904620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a5197dd4-46ae-46f5-b265-3899d351a615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:07.939930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 189/390] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:07.976613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec163f-7a9f-4d81-8d33-a1eab61b5589 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.018161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 202/391] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:08.065390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5197dd4-46ae-46f5-b265-3899d351a615 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.096882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:08.121577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:08.135784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec163f-7a9f-4d81-8d33-a1eab61b5589 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.180955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5197dd4-46ae-46f5-b265-3899d351a615 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.191682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.193005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.193239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.193817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.211159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:08.218871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.219855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.220356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.220857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:08.223309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:08.224284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:08.225111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 190/392] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:08.230670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.244842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:08.251686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:08.252616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:08.253485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 203/393] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:08.259393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.355505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5edabc-63e7-4bff-8cf5-4581fa0f569d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.391572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:08.402051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.403290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.403783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.404339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.409383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:08.409933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:08.412288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:08.425910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 191/394] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:13:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:08.432067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.464069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707c4a9-9c95-4a2a-a451-be9da0b0929e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.475215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:08.522645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.523516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.523939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.524396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.525359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:08.525876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:08.527395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:08.544393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 204/395] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:08.549543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.550063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8976cd-9e29-488e-906c-4ca28fb714b2 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.588004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.589008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.589514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.589989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.591369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:08.593140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:08.595234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 192/396] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:08 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:08.601258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.662272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:08.705892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4df88b-3900-4155-9f3d-4e580a0dae84 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.750113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.751142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.751630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.752139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.753589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:08.755347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:08.757339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 193/397] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:08 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:08.763438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.807415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5724b1-7a76-4c48-803c-70c7edb127bc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.875729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16229f-69df-4ce6-8e6e-6ee89526c87a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:08.899177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.899836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.900117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.900394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.901572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:08.903005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:08.907280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:08.911132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 205/398] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:08 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:08.916605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:08.918245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:08.919240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:08.919794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:08.920294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/groups/50f1b0a57d224fba8b9501122ca52499/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:08.922726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] RBAC: Authorizing `identity:create_grant(project_id=b518d31d96ad4d21b04a249253c6a4d5, group_id=50f1b0a57d224fba8b9501122ca52499, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:08.924415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:08.948698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 194/399] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 29 06:13:08 2022] PUT /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/groups/50f1b0a57d224fba8b9501122ca52499/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:13:08.984551 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:09.068866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7a811-5120-49e2-8823-b0f77015ecb4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:09.115370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2284df9-f293-4ab8-a31e-ea598a8331eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:09.116746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284df9-f293-4ab8-a31e-ea598a8331eb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:09.117258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284df9-f293-4ab8-a31e-ea598a8331eb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:09.117744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284df9-f293-4ab8-a31e-ea598a8331eb None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:09.119172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2284df9-f293-4ab8-a31e-ea598a8331eb None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:09.120953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2284df9-f293-4ab8-a31e-ea598a8331eb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:09.123261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 206/400] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:08 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:09.129749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:09.276672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052242f0-1b43-4f49-a493-44ca59bfd43a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:09.312781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bbcaf-98a6-4529-8315-58358f9daece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:09.313473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bbcaf-98a6-4529-8315-58358f9daece None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:09.313760 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bbcaf-98a6-4529-8315-58358f9daece None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:09.314040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bbcaf-98a6-4529-8315-58358f9daece None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/groups/50f1b0a57d224fba8b9501122ca52499/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:09.316388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bbcaf-98a6-4529-8315-58358f9daece None admin] RBAC: Authorizing `identity:create_grant(project_id=d90586d8e9094e65ad00907c4b6c8ddd, group_id=50f1b0a57d224fba8b9501122ca52499, role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:09.318206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bbcaf-98a6-4529-8315-58358f9daece None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:09.333654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 195/401] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 29 06:13:09 2022] PUT /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd/groups/50f1b0a57d224fba8b9501122ca52499/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:09.336909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc563-f33d-4573-b583-31d5f7a2410e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:09.337205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc563-f33d-4573-b583-31d5f7a2410e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:09.337492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc563-f33d-4573-b583-31d5f7a2410e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:09.339005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 207/402] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:09.345256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2879-4503-4536-948c-76da1c139814 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:09.345552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2879-4503-4536-948c-76da1c139814 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:09.345888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2879-4503-4536-948c-76da1c139814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:09.366828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9abc2879-4503-4536-948c-76da1c139814 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:09.368074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9abc2879-4503-4536-948c-76da1c139814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:09.420483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:09.461627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 196/403] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:09.484659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc563-f33d-4573-b583-31d5f7a2410e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:09.616247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc563-f33d-4573-b583-31d5f7a2410e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:09.623955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:09.653147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:09.653806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:09.654094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:09.654377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:09.666520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:09.672103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:09.672991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:09.673761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 208/404] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:09.679512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2879-4503-4536-948c-76da1c139814 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:09.784991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2879-4503-4536-948c-76da1c139814 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:09.821153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dece630-7b8f-4548-8e7a-c0fd0f3d2296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:09.821784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dece630-7b8f-4548-8e7a-c0fd0f3d2296 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:09.822060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dece630-7b8f-4548-8e7a-c0fd0f3d2296 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:09.822326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dece630-7b8f-4548-8e7a-c0fd0f3d2296 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:09.823330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dece630-7b8f-4548-8e7a-c0fd0f3d2296 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:09.823330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dece630-7b8f-4548-8e7a-c0fd0f3d2296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:09.824834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dece630-7b8f-4548-8e7a-c0fd0f3d2296 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:09.835126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 197/405] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:09.840885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:09.986602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46940fa-5a11-40e4-a4ba-9b1461c90048 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:10.030173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319de7d-3b2b-47d9-ae9f-a0080ec61b48 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.030710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319de7d-3b2b-47d9-ae9f-a0080ec61b48 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.030710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319de7d-3b2b-47d9-ae9f-a0080ec61b48 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.032287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 198/406] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:10.034935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:10.035798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.036336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.036934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.037465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08e44-7095-4140-aeb1-7c881b2c31cf None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.037758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08e44-7095-4140-aeb1-7c881b2c31cf None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.038040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08e44-7095-4140-aeb1-7c881b2c31cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.045353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:10.047133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:10.049010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 209/407] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:09 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:10.058113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.062465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e0f08e44-7095-4140-aeb1-7c881b2c31cf None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:10.063772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e0f08e44-7095-4140-aeb1-7c881b2c31cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:10.100783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:10.124961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:10.140770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 199/408] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:10.151174 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3e1d-7e07-472c-89d1-11542162fcb8 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.151766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3e1d-7e07-472c-89d1-11542162fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.152499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3e1d-7e07-472c-89d1-11542162fcb8 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.154309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 200/409] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:10.159775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa4a6-1efb-4f55-b26b-59881d527279 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.160169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa4a6-1efb-4f55-b26b-59881d527279 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.160575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa4a6-1efb-4f55-b26b-59881d527279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.171265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c028f2-4a68-4ff0-bdb7-be79980a4bde None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:10.190898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-bfaaa4a6-1efb-4f55-b26b-59881d527279 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:10.192393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bfaaa4a6-1efb-4f55-b26b-59881d527279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:10.207572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:10.209920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.210439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.210912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.212290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:10.213966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:10.216091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 210/410] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:10 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:10.217094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.278259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 201/411] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:10.279473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaa4a6-1efb-4f55-b26b-59881d527279 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.340959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4d6b-a43a-4d99-9aef-1507b820f37b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:10.342892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:10.380337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:10.381492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.382012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.382487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.394452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:10.401310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:10.402204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:10.403113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:10.404931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:10.406501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 211/412] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:10.408878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.415423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaa4a6-1efb-4f55-b26b-59881d527279 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:10.459275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:10.460286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.460599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.460882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/groups/50f1b0a57d224fba8b9501122ca52499/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.463522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b518d31d96ad4d21b04a249253c6a4d5, group_id=50f1b0a57d224fba8b9501122ca52499, role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:10.465328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:10.488565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 202/413] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 29 06:13:10 2022] PUT /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5/groups/50f1b0a57d224fba8b9501122ca52499/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:10.488565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.548910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381091-ef00-4160-a2fa-f23ba641889d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:10.605831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:10.607023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.607716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.608498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.636413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:10.646356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:10.647313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:10.648115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 212/414] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:10.658842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.704123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:10.741302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20259c-bcc4-4be1-bce4-5724362885c6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:10.784092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:10.785113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.785691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.786187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.787565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:10.788165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:10.789847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:10.806964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 203/415] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:10.810409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a713f-fda5-4240-a84a-4b71484a9255 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:10.812270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.850994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:10.871527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:10.872867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:10.877009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:10.877502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:10.878488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:10.879368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:10.881620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:10.898143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 213/416] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:13:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:10.904682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:10.975143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ee1c5-3c0a-473a-b512-513a88daa278 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:11.012553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:11.013670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.014151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.014621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] PATH_INFO: `/v3/groups/d347c15848b44f508d72002ad220c5c2` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.015989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] RBAC: Authorizing `identity:get_group(group_id=d347c15848b44f508d72002ad220c5c2)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:11.017857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:11.020073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 204/417] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:10 2022] GET /identity/v3/groups/d347c15848b44f508d72002ad220c5c2 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:11.025994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:11.099752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:11.122909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ec9f-7847-4039-bc83-0b4c0d0133b3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:11.160685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe8780-ed17-43ac-84a7-c3a88c52c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:11.161923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe8780-ed17-43ac-84a7-c3a88c52c72e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.162373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe8780-ed17-43ac-84a7-c3a88c52c72e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.162763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe8780-ed17-43ac-84a7-c3a88c52c72e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.198257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:11.241066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe8780-ed17-43ac-84a7-c3a88c52c72e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:11.242701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe8780-ed17-43ac-84a7-c3a88c52c72e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:11.244325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11c7d8-78a2-455e-904a-819d905acf3e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:11.295392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:11.296044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.296326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.296626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] PATH_INFO: `/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.298810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5535113d8e4690869b5d23e1b0a8b2)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:11.300242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:11.302579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 205/418] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:11 2022] GET /identity/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2 => generated 344 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:11.310372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:11.401398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:11.423864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 214/419] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:11.425230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25fd17-885c-4f95-b3b1-94c0802def91 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.425515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25fd17-885c-4f95-b3b1-94c0802def91 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.425791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25fd17-885c-4f95-b3b1-94c0802def91 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.427890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 215/420] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:11.433405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25fd17-885c-4f95-b3b1-94c0802def91 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:11.455139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc9f2-f300-484d-9f70-886e197a40a4 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:11.501106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252d98d-8172-408f-a7b7-80cfb49183da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:11.501792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252d98d-8172-408f-a7b7-80cfb49183da None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.502361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252d98d-8172-408f-a7b7-80cfb49183da None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.502645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252d98d-8172-408f-a7b7-80cfb49183da None admin] PATH_INFO: `/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2/groups/d347c15848b44f508d72002ad220c5c2/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.505284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252d98d-8172-408f-a7b7-80cfb49183da None admin] RBAC: Authorizing `identity:create_grant(project_id=cc5535113d8e4690869b5d23e1b0a8b2, group_id=d347c15848b44f508d72002ad220c5c2, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:11.507108 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252d98d-8172-408f-a7b7-80cfb49183da None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:11.528582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 206/421] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 29 06:13:11 2022] PUT /identity/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2/groups/d347c15848b44f508d72002ad220c5c2/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:11.528582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7917245-e45a-4af9-b726-db712be52608 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.528582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7917245-e45a-4af9-b726-db712be52608 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.528582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7917245-e45a-4af9-b726-db712be52608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.550428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f7917245-e45a-4af9-b726-db712be52608 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:11.552310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f7917245-e45a-4af9-b726-db712be52608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:11.569293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:11.613364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25fd17-885c-4f95-b3b1-94c0802def91 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:11.641778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 207/422] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:11.683216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7917245-e45a-4af9-b726-db712be52608 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:11.691113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:11.692034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.692816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.693370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.697908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:11.699746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:11.710514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 216/423] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:11 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:11.717069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:11.806950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:11.836714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7917245-e45a-4af9-b726-db712be52608 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:11.880952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e668f-786a-4eca-8af2-daf929147f28 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:11.882995 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cb63b-a73e-4a87-8fbd-c81b98c9dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:11.884635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cb63b-a73e-4a87-8fbd-c81b98c9dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.884926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cb63b-a73e-4a87-8fbd-c81b98c9dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.885216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cb63b-a73e-4a87-8fbd-c81b98c9dc8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.899474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cb63b-a73e-4a87-8fbd-c81b98c9dc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-8b8cb63b-a73e-4a87-8fbd-c81b98c9dc8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:11.906208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:11.907087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 06:13:11.907856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:11.908437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 208/424] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:11.909600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c673fdb-7050-417b-816c-49d7fa7903a2 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.912566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c673fdb-7050-417b-816c-49d7fa7903a2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.912566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c673fdb-7050-417b-816c-49d7fa7903a2 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.914244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 209/425] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:11.915054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c673fdb-7050-417b-816c-49d7fa7903a2 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:11.929555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:11.937637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e637e29-8734-48c7-905b-f5ef05dbe14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:11.938474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e637e29-8734-48c7-905b-f5ef05dbe14f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.938763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e637e29-8734-48c7-905b-f5ef05dbe14f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.939044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e637e29-8734-48c7-905b-f5ef05dbe14f None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.940658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e637e29-8734-48c7-905b-f5ef05dbe14f None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:11.942157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e637e29-8734-48c7-905b-f5ef05dbe14f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:11.945617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 217/426] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:11 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:11.947600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1790c-0c36-4b66-8d20-92c5045e820d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:11.948035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1790c-0c36-4b66-8d20-92c5045e820d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:11.948443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1790c-0c36-4b66-8d20-92c5045e820d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:11.976456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7fc1790c-0c36-4b66-8d20-92c5045e820d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:11.977702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7fc1790c-0c36-4b66-8d20-92c5045e820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:12.030426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c673fdb-7050-417b-816c-49d7fa7903a2 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.068999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.069721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.070076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.070421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.071260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:12.071639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:12.073363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:12.076623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 218/427] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:12.077536 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1790c-0c36-4b66-8d20-92c5045e820d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.094481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 210/428] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:13:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:12.101948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.162301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:12.179112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:12.214793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d89ff-512e-4d8b-adce-c5b47bffacf9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.225899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1790c-0c36-4b66-8d20-92c5045e820d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.254429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.255803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.256225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.256667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.262885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.263512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.263799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.264082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.264900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:12.265213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:12.266660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:12.274374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 219/429] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:12.275266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.339111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:12.340999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:12.387914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66e386-1441-44e0-909d-781c2e35228d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.403702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:12.424142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:12.429106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.429940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.430228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.430509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.445364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:12.451137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:12.452042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:12.453276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:12.456820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 220/430] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:12.459518 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.467985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 211/431] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 06:13:12.481613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.589073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f2b41-77ae-4b25-960a-07f334248d93 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.632277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.633042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.633346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.633592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.634367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:12.634682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:12.636381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:12.639420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ac3b4-a67d-4b8d-9243-0dc6a2ab9db9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.648285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 221/432] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:12.653781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.681017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.682044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.682519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.682983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.684602 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:12.686232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:12.688247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 212/433] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:12.698831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.703651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:12.745597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:12.783283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca266a-79a3-4005-98bd-3cf57b3b25e5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.809551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6c55e-e067-4138-8bb6-0c2d76f417a8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:12.822463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d894df-52aa-404f-bca5-ece8d4eff42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.823411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d894df-52aa-404f-bca5-ece8d4eff42f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.823913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d894df-52aa-404f-bca5-ece8d4eff42f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.824407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d894df-52aa-404f-bca5-ece8d4eff42f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.848118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:12.848838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:12.849119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:12.849397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:12.850700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:12.852143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:12.854115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 213/434] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:12.862343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:12.933642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d894df-52aa-404f-bca5-ece8d4eff42f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:12.935118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d894df-52aa-404f-bca5-ece8d4eff42f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:12.983222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:13.017260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017afbfd-6040-476e-a5ce-e3435515f591 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:13.026046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:13.063020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7410-3cfe-4fab-bb74-910bfbb31432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:13.063898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7410-3cfe-4fab-bb74-910bfbb31432 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.064184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7410-3cfe-4fab-bb74-910bfbb31432 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.064462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7410-3cfe-4fab-bb74-910bfbb31432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.065389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7410-3cfe-4fab-bb74-910bfbb31432 None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=5f97bcaf49fe487d8def3bf0991dcf15, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:13.065703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7410-3cfe-4fab-bb74-910bfbb31432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:13.065861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 222/435] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:13.067101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4c109-a222-4c0e-819c-5f7703df09dd None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.067266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7410-3cfe-4fab-bb74-910bfbb31432 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:13.067413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4c109-a222-4c0e-819c-5f7703df09dd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.067659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4c109-a222-4c0e-819c-5f7703df09dd None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.069373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 223/436] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:13.071342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4c109-a222-4c0e-819c-5f7703df09dd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:13.078541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 214/437] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:12 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=5f97bcaf49fe487d8def3bf0991dcf15 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:13.080960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99025d7-d5cc-4072-8e6f-3f368e721ed1 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.081073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99025d7-d5cc-4072-8e6f-3f368e721ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.081710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99025d7-d5cc-4072-8e6f-3f368e721ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.106839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b99025d7-d5cc-4072-8e6f-3f368e721ed1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:13.109318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b99025d7-d5cc-4072-8e6f-3f368e721ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:13.200185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4c109-a222-4c0e-819c-5f7703df09dd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:13.204481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 215/438] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:13.232025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99025d7-d5cc-4072-8e6f-3f368e721ed1 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:13.245728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:13.246347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.246633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.246908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] PATH_INFO: `/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.248113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] RBAC: Authorizing `identity:get_project(project_id=d90586d8e9094e65ad00907c4b6c8ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:13.249614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:13.251600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 224/439] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/projects/d90586d8e9094e65ad00907c4b6c8ddd => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:13.257599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:13.262681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:13.364564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:13.368085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99025d7-d5cc-4072-8e6f-3f368e721ed1 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:13.386125 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fea0ed-6a6a-402f-8824-3ab76bdde700 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:13.407261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:13.407964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.408247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.408983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.419003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:13.423656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:13.423901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:13.425034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:13.431437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:13.432042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.432042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.432042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.432222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 216/440] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:13.432222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:13.432338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:13.432760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:13.434756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 225/441] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:13.440313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:13.515890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:13.543653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebde6f-e0ec-4e97-81a3-e3e9b3767d53 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:13.549336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cd6eb-0b84-445b-9a41-26f1f154aaf1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:13.579412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:13.580033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.580316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.580878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.581683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:13.581989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:13.583751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:13.587918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5940f3-8d22-45ce-8d26-7b1e714bfe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:13.589127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5940f3-8d22-45ce-8d26-7b1e714bfe1f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.589515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5940f3-8d22-45ce-8d26-7b1e714bfe1f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.589891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5940f3-8d22-45ce-8d26-7b1e714bfe1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.590766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5940f3-8d22-45ce-8d26-7b1e714bfe1f None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:13.591229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5940f3-8d22-45ce-8d26-7b1e714bfe1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:13.592017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 217/442] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:13.592861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5940f3-8d22-45ce-8d26-7b1e714bfe1f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:13.597271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:13.599827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 226/443] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=d90586d8e9094e65ad00907c4b6c8ddd&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:13.626164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:13.711675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bf8ee-877e-435b-a2d0-2d1e83501038 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:13.752871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:13.753650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.754044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.754427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.826729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:13.828193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:13.849058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a77f-3dd6-4823-b280-553a78573a29 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.849629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a77f-3dd6-4823-b280-553a78573a29 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.849629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a77f-3dd6-4823-b280-553a78573a29 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.850963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 227/444] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:13.854563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:13.856487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9491-b7cb-477a-8235-35856733c9b0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:13.856844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9491-b7cb-477a-8235-35856733c9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:13.857194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9491-b7cb-477a-8235-35856733c9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:13.877685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-525f9491-b7cb-477a-8235-35856733c9b0 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:13.878811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-525f9491-b7cb-477a-8235-35856733c9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:13.946070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 218/445] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 06:13:13.951511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:13.968585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 228/446] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:13.991682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f9491-b7cb-477a-8235-35856733c9b0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:14.072248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0e0fd-3e7c-45b0-a93a-f830633be82d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.073804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:14.102953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:14.118129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f9491-b7cb-477a-8235-35856733c9b0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.121164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.121821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.122109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.122389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] PATH_INFO: `/v3/projects/b518d31d96ad4d21b04a249253c6a4d5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.123558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=b518d31d96ad4d21b04a249253c6a4d5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:14.125030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:14.126698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 219/447] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/projects/b518d31d96ad4d21b04a249253c6a4d5 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:14.133031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:14.166806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.167447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.167730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.168008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.178606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:14.183656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:14.184743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:14.190615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 229/448] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:14.191432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:14.243856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c138f-453f-471c-b1c5-a8dae1db7ea9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.277044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.277680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.277960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.278231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] PATH_INFO: `/v3/groups/50f1b0a57d224fba8b9501122ca52499` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.279786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] RBAC: Authorizing `identity:get_group(group_id=50f1b0a57d224fba8b9501122ca52499)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:14.281264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:14.282917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 220/449] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:14 2022] GET /identity/v3/groups/50f1b0a57d224fba8b9501122ca52499 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:14.289321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:14.291536 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b172ee3-c33f-49eb-8030-6624cbf6cb24 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.316439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:14.325959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.326572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.326856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.327129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.327834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:14.328141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:14.329561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:14.338651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 230/450] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:14.344115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:14.347868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:14.391285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b89bb-4dd1-4712-82a7-51dd78fc5f7c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.425282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.425911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.426198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.426528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.427468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] RBAC: Adding query filter params (group.id=50f1b0a57d224fba8b9501122ca52499, role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=b518d31d96ad4d21b04a249253c6a4d5) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:14.427785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:14.429237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:14.434921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 221/451] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:14 2022] GET /identity/v3/role_assignments?group.id=50f1b0a57d224fba8b9501122ca52499&scope.project.id=b518d31d96ad4d21b04a249253c6a4d5&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:14.448425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10194054-00a0-49cf-b4c9-a55927247481 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.485812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.486453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.486735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.487017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.546254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:14.559347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:14.560845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:14.661643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 231/452] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 06:13:14.667201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:14.775407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8b681-772f-4bfd-83f4-56193e9224f4 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.797363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:14.809534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.810179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.810466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.810744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] PATH_INFO: `/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.811896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] RBAC: Authorizing `identity:get_project(project_id=cc5535113d8e4690869b5d23e1b0a8b2)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:14.813355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:14.814999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 222/453] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:13:14 2022] GET /identity/v3/projects/cc5535113d8e4690869b5d23e1b0a8b2 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:14.821561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:14.905615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:14.917628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c4eed-ad34-4442-ae27-e90d1f3a1c8a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:14.951026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d43cc4-2762-4e68-a245-90c0881242f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:14.951653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d43cc4-2762-4e68-a245-90c0881242f5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:14.951938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d43cc4-2762-4e68-a245-90c0881242f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:14.952217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d43cc4-2762-4e68-a245-90c0881242f5 None admin] PATH_INFO: `/v3/groups/d347c15848b44f508d72002ad220c5c2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:14.953391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d43cc4-2762-4e68-a245-90c0881242f5 None admin] RBAC: Authorizing `identity:get_group(group_id=d347c15848b44f508d72002ad220c5c2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:14.954819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d43cc4-2762-4e68-a245-90c0881242f5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:14.956665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 232/454] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 29 06:13:14 2022] GET /identity/v3/groups/d347c15848b44f508d72002ad220c5c2 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:14.962723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:15.062201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9cd6f-b8c8-4e21-ac6b-68faf3d5542b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:15.094242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b13c37-ff24-4605-b449-3631c8d84ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:15.094852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13c37-ff24-4605-b449-3631c8d84ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:15.095141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13c37-ff24-4605-b449-3631c8d84ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:15.095421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13c37-ff24-4605-b449-3631c8d84ec5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:15.096199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b13c37-ff24-4605-b449-3631c8d84ec5 None admin] RBAC: Adding query filter params (group.id=d347c15848b44f508d72002ad220c5c2, role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=cc5535113d8e4690869b5d23e1b0a8b2) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:15.096514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b13c37-ff24-4605-b449-3631c8d84ec5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:15.097971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b13c37-ff24-4605-b449-3631c8d84ec5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:15.104089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 223/455] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 29 06:13:14 2022] GET /identity/v3/role_assignments?group.id=d347c15848b44f508d72002ad220c5c2&scope.project.id=cc5535113d8e4690869b5d23e1b0a8b2&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:17.507052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e928-3d1e-4cee-9ee5-e30f35722a95 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:17.507964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e928-3d1e-4cee-9ee5-e30f35722a95 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:17.508387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e928-3d1e-4cee-9ee5-e30f35722a95 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:17.510152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 233/456] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:17.522835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68aa4d-dd4e-4091-b6e4-cac92ab25021 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:17.523309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68aa4d-dd4e-4091-b6e4-cac92ab25021 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:17.523397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68aa4d-dd4e-4091-b6e4-cac92ab25021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:17.549555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ba68aa4d-dd4e-4091-b6e4-cac92ab25021 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:17.550765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ba68aa4d-dd4e-4091-b6e4-cac92ab25021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:17.622792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a212007-3094-431f-ad38-c8e27a6215c8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:17.623085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a212007-3094-431f-ad38-c8e27a6215c8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:17.623369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a212007-3094-431f-ad38-c8e27a6215c8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:17.625008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 234/457] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:17.629050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:17.632013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ae79-41a4-4a5a-a49c-23b735a581ed None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:17.632305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ae79-41a4-4a5a-a49c-23b735a581ed None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:17.632635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ae79-41a4-4a5a-a49c-23b735a581ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:17.644350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 224/458] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:17.660805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6903ae79-41a4-4a5a-a49c-23b735a581ed None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:17.662401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6903ae79-41a4-4a5a-a49c-23b735a581ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:17.685549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68aa4d-dd4e-4091-b6e4-cac92ab25021 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:17.782611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 235/459] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:17.816605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903ae79-41a4-4a5a-a49c-23b735a581ed None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:17.839197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68aa4d-dd4e-4091-b6e4-cac92ab25021 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:17.885836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:17.886489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:17.886771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:17.887045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:17.907770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:17.909482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:17.938587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:17.939699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:17.940624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:17.940624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:17.940624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:17.940624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:17.940624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 225/460] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:17.947012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:17.988130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:18.012090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:18.013449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903ae79-41a4-4a5a-a49c-23b735a581ed None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.057286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7d9e6-6a35-420c-ac24-967a3ac8324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.058221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d9e6-6a35-420c-ac24-967a3ac8324d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.058674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d9e6-6a35-420c-ac24-967a3ac8324d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.059117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d9e6-6a35-420c-ac24-967a3ac8324d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.075152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7d9e6-6a35-420c-ac24-967a3ac8324d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:18.076901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7d9e6-6a35-420c-ac24-967a3ac8324d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-07e7d9e6-6a35-420c-ac24-967a3ac8324d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:18.094520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:18.095422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:18.096254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:18.096254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:18.096254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:18.096254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:18.098678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 236/461] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:18.105276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b1cf-4d96-4933-9fc3-316eec8416b4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.105962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b1cf-4d96-4933-9fc3-316eec8416b4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.106447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b1cf-4d96-4933-9fc3-316eec8416b4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.107771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c8f8-7b2b-4b23-9f51-ac03026f7a3e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.108892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 237/462] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:18.109945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9b1cf-4d96-4933-9fc3-316eec8416b4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:18.150664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bcdd4-f2ea-4e63-8ddb-705492282163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.151812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bcdd4-f2ea-4e63-8ddb-705492282163 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.152923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bcdd4-f2ea-4e63-8ddb-705492282163 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.153607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bcdd4-f2ea-4e63-8ddb-705492282163 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.154863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bcdd4-f2ea-4e63-8ddb-705492282163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:18.155653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bcdd4-f2ea-4e63-8ddb-705492282163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:18.157852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bcdd4-f2ea-4e63-8ddb-705492282163 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:18.174697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 226/463] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:18.176662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad9ca1-5442-4011-b07f-f6301e462c40 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.177076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad9ca1-5442-4011-b07f-f6301e462c40 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.177505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad9ca1-5442-4011-b07f-f6301e462c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.202769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-96ad9ca1-5442-4011-b07f-f6301e462c40 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:18.205164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-96ad9ca1-5442-4011-b07f-f6301e462c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:18.246769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9b1cf-4d96-4933-9fc3-316eec8416b4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.277889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 227/464] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:18.279082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad9ca1-5442-4011-b07f-f6301e462c40 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:18.296506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.297353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.297798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.298622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.299467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:18.299915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:18.301673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:18.310942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:18.319331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 238/465] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:18.321566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:18.328117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:18.423082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad9ca1-5442-4011-b07f-f6301e462c40 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.434509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ef33-56b9-4ce5-a419-fd7bda878546 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.463961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.464704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.465312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.465590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.466410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:18.467828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:18.471050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee000df-2675-493d-845d-b46051347275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.471697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee000df-2675-493d-845d-b46051347275 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.471986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee000df-2675-493d-845d-b46051347275 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.472278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee000df-2675-493d-845d-b46051347275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.483605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee000df-2675-493d-845d-b46051347275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:18.485114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee000df-2675-493d-845d-b46051347275 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:18.496573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 228/466] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:13:18 2022] POST /identity/v3/users => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:18.496573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-eee000df-2675-493d-845d-b46051347275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:18.513217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:18.514471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:18.515334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:18.515334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:18.515334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:18.515334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:18.516885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 239/467] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:18.518042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab8811-e3ff-461f-98e9-ca6181539c69 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.518334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab8811-e3ff-461f-98e9-ca6181539c69 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.518620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab8811-e3ff-461f-98e9-ca6181539c69 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.520679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 240/468] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:18.521485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab8811-e3ff-461f-98e9-ca6181539c69 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:18.596782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:18.620155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:18.646275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef605c4c-cdde-4bdc-8f35-d8ddec45865a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.673992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab8811-e3ff-461f-98e9-ca6181539c69 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.696713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.697356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.697637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.697912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.698683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:18.700642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:18.718243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91921a9f-6982-4f94-96c8-e028f8a6680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.719137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921a9f-6982-4f94-96c8-e028f8a6680c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.719554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921a9f-6982-4f94-96c8-e028f8a6680c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.719880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921a9f-6982-4f94-96c8-e028f8a6680c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.720618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91921a9f-6982-4f94-96c8-e028f8a6680c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:18.720983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91921a9f-6982-4f94-96c8-e028f8a6680c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:18.722372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91921a9f-6982-4f94-96c8-e028f8a6680c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:18.737747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 241/469] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:18.739433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89999-3c4c-42dd-af5f-51f0afbc39f5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:18.739433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89999-3c4c-42dd-af5f-51f0afbc39f5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:18.739433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89999-3c4c-42dd-af5f-51f0afbc39f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:18.740369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 229/470] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:13:18 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:18.749627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:18.761549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1eb89999-3c4c-42dd-af5f-51f0afbc39f5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:18.762770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1eb89999-3c4c-42dd-af5f-51f0afbc39f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:18.847937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 242/471] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:18.874239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb89999-3c4c-42dd-af5f-51f0afbc39f5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:18.922234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:18.954797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158cebb-72c9-4044-9075-9d70a80a2150 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.987632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb89999-3c4c-42dd-af5f-51f0afbc39f5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:18.999216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:18.999858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.000403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.000517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.001364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:19.002780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:19.015815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:19.027880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d1be8-25a3-441c-8c54-4fe95d0a227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:19.028774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d1be8-25a3-441c-8c54-4fe95d0a227b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.029226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d1be8-25a3-441c-8c54-4fe95d0a227b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.029638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d1be8-25a3-441c-8c54-4fe95d0a227b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.032788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 230/472] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:13:18 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:19.042285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d1be8-25a3-441c-8c54-4fe95d0a227b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:19.044073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d1be8-25a3-441c-8c54-4fe95d0a227b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-874d1be8-25a3-441c-8c54-4fe95d0a227b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:19.061818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:19.062889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:19.063937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:19.063937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:19.063937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:19.063937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:19.065580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 243/473] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:19.072027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:19.156169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24846a5-6262-4bee-b1d3-c9710280e7e5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.156745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24846a5-6262-4bee-b1d3-c9710280e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.157228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24846a5-6262-4bee-b1d3-c9710280e7e5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.159044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 244/474] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:19.172286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fa202-f707-4d6d-ba47-8a443f7021eb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.172764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fa202-f707-4d6d-ba47-8a443f7021eb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.173147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fa202-f707-4d6d-ba47-8a443f7021eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.203258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-012fa202-f707-4d6d-ba47-8a443f7021eb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:19.205046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-012fa202-f707-4d6d-ba47-8a443f7021eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:19.209299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0d268-7bab-4c6d-8aa8-d879dfa99045 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:19.254582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:19.255494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.255786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.256094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.257780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:19.258094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:19.259610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:19.275744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 231/475] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:19.282823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:19.301884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 245/476] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:19.325559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fa202-f707-4d6d-ba47-8a443f7021eb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:19.352163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:19.364994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:19.403158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842de4-1228-4ad5-9ec0-816a7d28653a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:19.438733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fa202-f707-4d6d-ba47-8a443f7021eb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:19.445704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:19.446352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.446645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.446934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.447743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:19.450020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:19.479352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 232/477] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:13:19 2022] POST /identity/v3/users => generated 249 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:19.490842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:19.491576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.491860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.492238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.493106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:19.494488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:19.506022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.sql.core [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 06:13:19.506022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 06:13:19.506022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [parameters: {'id': '15d4f24744c24624853c0ad2b73db561', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 06:13:19.506022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80161) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() Jul 29 06:13:19.527382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return ref.to_dict() Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 06:13:19.528760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application yield resource Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise self.value Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application session.commit() Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 06:13:19.530381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.flush() Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application compat.raise_( Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 06:13:19.532485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 06:13:19.534682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 06:13:19.537453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [parameters: {'id': '15d4f24744c24624853c0ad2b73db561', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:19.539379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 06:13:19.548922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:19.551043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 246/478] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 29 06:13:19 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:19.551218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:19.672196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574867-4bd5-4ab7-9f4d-358b0426ebed None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:19.684652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:19.714049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e207f49-123e-404d-aad3-17800c3e245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:19.715050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e207f49-123e-404d-aad3-17800c3e245e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.715532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e207f49-123e-404d-aad3-17800c3e245e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.716003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e207f49-123e-404d-aad3-17800c3e245e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.727542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e207f49-123e-404d-aad3-17800c3e245e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-6e207f49-123e-404d-aad3-17800c3e245e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:19.733678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:19.734488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:19.735257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 233/479] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:19.740701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:19.842642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:19.859787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0149086-c8cb-4aa9-b123-f725cbb1c21c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:19.897424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f07ae3-f945-460c-8673-68f8cc020f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:19.898079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07ae3-f945-460c-8673-68f8cc020f22 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:19.898366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07ae3-f945-460c-8673-68f8cc020f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:19.898648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07ae3-f945-460c-8673-68f8cc020f22 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:19.899394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f07ae3-f945-460c-8673-68f8cc020f22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:19.899707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f07ae3-f945-460c-8673-68f8cc020f22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:19.901223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f07ae3-f945-460c-8673-68f8cc020f22 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:19.910936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 247/480] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:20.881984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ea9c7-6f3f-4a7b-ba99-5ec508d2573f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:20.882490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ea9c7-6f3f-4a7b-ba99-5ec508d2573f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:20.882596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ea9c7-6f3f-4a7b-ba99-5ec508d2573f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:20.884236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 234/481] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:20.889990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f9526-a3f2-4a68-b8b3-1b3551888d08 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:20.890395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f9526-a3f2-4a68-b8b3-1b3551888d08 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:20.890550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f9526-a3f2-4a68-b8b3-1b3551888d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:20.921547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d82f9526-a3f2-4a68-b8b3-1b3551888d08 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:20.922976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d82f9526-a3f2-4a68-b8b3-1b3551888d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:21.025013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 248/482] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:21.067765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ea9c7-6f3f-4a7b-ba99-5ec508d2573f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:21.218189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ea9c7-6f3f-4a7b-ba99-5ec508d2573f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:21.260000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301bdba-ec91-4cac-a71c-e72b738a423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:21.260683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301bdba-ec91-4cac-a71c-e72b738a423e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.260971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301bdba-ec91-4cac-a71c-e72b738a423e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.261251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301bdba-ec91-4cac-a71c-e72b738a423e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.272752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301bdba-ec91-4cac-a71c-e72b738a423e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-7301bdba-ec91-4cac-a71c-e72b738a423e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:21.281713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:21.282923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:21.283745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 235/483] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:21.289645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f9526-a3f2-4a68-b8b3-1b3551888d08 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:21.394561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:21.429632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f9526-a3f2-4a68-b8b3-1b3551888d08 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:21.475576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55889-1032-4c43-aa9d-62aaa617e8e6 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.475876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55889-1032-4c43-aa9d-62aaa617e8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.476142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55889-1032-4c43-aa9d-62aaa617e8e6 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.480560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 236/484] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:21.483373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c29f9-77f2-4fb9-bdfe-8c175af47acd None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.483772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c29f9-77f2-4fb9-bdfe-8c175af47acd None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.484060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c29f9-77f2-4fb9-bdfe-8c175af47acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.484866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:21.485494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.485774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.486045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.486759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:21.487061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:21.488429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:21.505504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f84c29f9-77f2-4fb9-bdfe-8c175af47acd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:21.506868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f84c29f9-77f2-4fb9-bdfe-8c175af47acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:21.507313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 249/485] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:21.512519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:21.534347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:21.592645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 237/486] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:21.611876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c29f9-77f2-4fb9-bdfe-8c175af47acd None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:21.729304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c29f9-77f2-4fb9-bdfe-8c175af47acd None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:21.769529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:21.770926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.771207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f2e3b-59d6-4c3d-8c6d-efac8ee0d127 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:21.771318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.771498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.783823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:21.789351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:21.790231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:21.791006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:21.795573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 238/487] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:21.796777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:21.821452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:21.849577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd5986-9a4f-44e2-93df-ef189291ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:21.850236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd5986-9a4f-44e2-93df-ef189291ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.850516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd5986-9a4f-44e2-93df-ef189291ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.850796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd5986-9a4f-44e2-93df-ef189291ddf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.860179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:21.864225 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd5986-9a4f-44e2-93df-ef189291ddf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:21.866034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd5986-9a4f-44e2-93df-ef189291ddf9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3fdd5986-9a4f-44e2-93df-ef189291ddf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:21.881971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:21.882867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:21.883700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:21.883700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:21.883700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:21.883700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:21.885994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 250/488] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:21.887614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bb684-d642-4008-a819-3ecce28bf558 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.888093 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bb684-d642-4008-a819-3ecce28bf558 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.888608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bb684-d642-4008-a819-3ecce28bf558 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.890338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 251/489] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:21.891766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bb684-d642-4008-a819-3ecce28bf558 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:21.913673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379be7-f672-42ce-b0ba-f3dcd5b1655c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:21.960610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36acbaf-8239-42cb-b292-22a97c9c1ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:21.961251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36acbaf-8239-42cb-b292-22a97c9c1ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.961545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36acbaf-8239-42cb-b292-22a97c9c1ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.961816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36acbaf-8239-42cb-b292-22a97c9c1ab2 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:21.962537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36acbaf-8239-42cb-b292-22a97c9c1ab2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:21.962870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36acbaf-8239-42cb-b292-22a97c9c1ab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:21.965031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36acbaf-8239-42cb-b292-22a97c9c1ab2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:21.980627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 239/490] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:21.982260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7552e6c-9af2-4c11-9cb8-c60112cc0dee None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:21.982545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7552e6c-9af2-4c11-9cb8-c60112cc0dee None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:21.982822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7552e6c-9af2-4c11-9cb8-c60112cc0dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.001807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c7552e6c-9af2-4c11-9cb8-c60112cc0dee None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:22.003286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c7552e6c-9af2-4c11-9cb8-c60112cc0dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:22.030987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bb684-d642-4008-a819-3ecce28bf558 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.077368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 240/491] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:22.078210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7552e6c-9af2-4c11-9cb8-c60112cc0dee None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.078945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.079829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.080873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.082295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.083803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:22.084316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:22.085965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:22.094096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.107861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 252/492] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:22.115504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.174996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.188273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7552e6c-9af2-4c11-9cb8-c60112cc0dee None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.226539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.227444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.227730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.228148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.229951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eb324-adc8-44d5-908f-257e4614b065 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.238429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:22.241164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:22.263426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:22.264300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:22.265208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:22.265208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:22.265208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:22.265208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:22.267967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea48979-4155-402d-83d9-f63de49c2215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.268636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea48979-4155-402d-83d9-f63de49c2215 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.268862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 241/493] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:22.268957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea48979-4155-402d-83d9-f63de49c2215 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.269179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea48979-4155-402d-83d9-f63de49c2215 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.270507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.277023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea48979-4155-402d-83d9-f63de49c2215 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:22.278484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea48979-4155-402d-83d9-f63de49c2215 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8ea48979-4155-402d-83d9-f63de49c2215 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:22.293559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:22.294484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 253/494] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a89fc-dc3f-4525-9bfe-bb6b0a61af96 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.300847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a89fc-dc3f-4525-9bfe-bb6b0a61af96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.302239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a89fc-dc3f-4525-9bfe-bb6b0a61af96 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.302239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 254/495] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:22.302239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a89fc-dc3f-4525-9bfe-bb6b0a61af96 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.344496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.387131 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0939f-6ea3-489c-b293-ee73f15829dc None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.414159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a89fc-dc3f-4525-9bfe-bb6b0a61af96 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.429562 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.429844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.430156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.430606 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.433317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.444880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:22.450407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:22.451488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:22.452289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 242/496] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:22.453031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.465448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d67cd-10ac-42a6-a176-a5326c5f7159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.466281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d67cd-10ac-42a6-a176-a5326c5f7159 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.466567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d67cd-10ac-42a6-a176-a5326c5f7159 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.466845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d67cd-10ac-42a6-a176-a5326c5f7159 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.467528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d67cd-10ac-42a6-a176-a5326c5f7159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:22.467831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d67cd-10ac-42a6-a176-a5326c5f7159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:22.469241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d67cd-10ac-42a6-a176-a5326c5f7159 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:22.481510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 255/497] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:22.482530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f065a-5a5a-4e4e-80f6-4824fc9a10da None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.482816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f065a-5a5a-4e4e-80f6-4824fc9a10da None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.483094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f065a-5a5a-4e4e-80f6-4824fc9a10da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.501181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-999f065a-5a5a-4e4e-80f6-4824fc9a10da None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:22.502358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-999f065a-5a5a-4e4e-80f6-4824fc9a10da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:22.565146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ca330-53fa-4b2d-b277-905d6094d344 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.577825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 256/498] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:22.579000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f065a-5a5a-4e4e-80f6-4824fc9a10da None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.597344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.608905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.609533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.609816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.610099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.610892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:22.611206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:22.612662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:22.648976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 243/499] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:22.649628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.684896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f065a-5a5a-4e4e-80f6-4824fc9a10da None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.686537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.725943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174bdb7-871b-4098-9ac6-b55bd2180bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.726865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174bdb7-871b-4098-9ac6-b55bd2180bef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.729921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174bdb7-871b-4098-9ac6-b55bd2180bef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.730405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174bdb7-871b-4098-9ac6-b55bd2180bef None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.731495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174bdb7-871b-4098-9ac6-b55bd2180bef None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:22.731968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174bdb7-871b-4098-9ac6-b55bd2180bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:22.733538 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174bdb7-871b-4098-9ac6-b55bd2180bef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:22.745156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 257/500] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:22.747132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf00d45-3067-4a5f-81bc-133db198fe8c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.748011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf00d45-3067-4a5f-81bc-133db198fe8c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.748470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf00d45-3067-4a5f-81bc-133db198fe8c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.750384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 258/501] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:22.751252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf00d45-3067-4a5f-81bc-133db198fe8c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.760824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424640d-cb56-4ab0-9ab2-1a49589fc24d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.797221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.797867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.798150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.798485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.807742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:22.808651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:22.822696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:22.823595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 06:13:22.824451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:22.825845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 244/502] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:22.826639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.868236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf00d45-3067-4a5f-81bc-133db198fe8c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.876015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.904463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.905142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.905429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.905711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:22.916812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:22.922357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:22.923215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:22.924256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 259/503] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:22.925752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:22.942271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb96a18-e325-4964-a6ba-7db1705eb346 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:22.960268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:22.981028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:22.981938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:22.982227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:22.982506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.043934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969f151-3409-4536-9724-7bfb21c47cae None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.054636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.056115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.077114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9da6ca-5b95-44ef-a8dd-fb92423564f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.077755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9da6ca-5b95-44ef-a8dd-fb92423564f1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.078040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9da6ca-5b95-44ef-a8dd-fb92423564f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.078318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9da6ca-5b95-44ef-a8dd-fb92423564f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.090775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9da6ca-5b95-44ef-a8dd-fb92423564f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.092321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9da6ca-5b95-44ef-a8dd-fb92423564f1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.097300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1b9da6ca-5b95-44ef-a8dd-fb92423564f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:23.107432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:23.108380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:23.109284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:23.109284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:23.109284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.109284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:23.109284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 260/504] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:23.110044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668b92d-b70b-4ce2-8159-5e1547b84687 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.110332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668b92d-b70b-4ce2-8159-5e1547b84687 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.110614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668b92d-b70b-4ce2-8159-5e1547b84687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.128166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e668b92d-b70b-4ce2-8159-5e1547b84687 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:23.129841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e668b92d-b70b-4ce2-8159-5e1547b84687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:23.161319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 245/505] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:23.162168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.198980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 261/506] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:23.199809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668b92d-b70b-4ce2-8159-5e1547b84687 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.220427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.264778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342f842-d853-420e-b225-cf4a3079eb31 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.298391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.299004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.299284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.299558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.300336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:23.300674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.302106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.312972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668b92d-b70b-4ce2-8159-5e1547b84687 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.315372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.319666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 246/507] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:23.320722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.349832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.350457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.350739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.351016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.351719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:23.352019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.353424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.361513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 262/508] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:23.362279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.424105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0eee-fbe5-48da-b9c3-fbc15ffe4329 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.428524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.460509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.461154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.461441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.461720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.462409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:23.462717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.463645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33f31d-8b9e-4ac8-9033-52aa9d2de2c9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.464092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.475686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 247/509] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:23.476474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.497729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.498368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.498651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.498945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.510352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.511779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:23.526982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:23.527829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:23.528689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:23.528689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:23.528689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.528689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:23.528689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 263/510] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:23.529208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.538910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.580929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4286d1e-5376-4f78-894c-b32adee1c85c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.614298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.615080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.615364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.615642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.626329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.627781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.632059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a80df7-4444-4eaf-a0f7-71eff5d7f862 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:23.643775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:23.644642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:23.646241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:23.646241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:23.646241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.646241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:23.646241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 248/511] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:23.647765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.654001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.668708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.669332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.669616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.669894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.740008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.741500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.750281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5f768-03e9-423a-ac49-de0966474142 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.763671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.785241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.785885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.786174 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.786454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.796200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.797682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:23.812611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:23.813660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:23.814727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:23.814727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:23.814727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:23.814727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:23.814727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 249/512] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:23.815206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.841838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 264/513] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:23.842675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.862822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:23.920627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e0403-233e-4cb3-bea2-40d957081f5a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.946063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e6b24-17bc-4b84-8b48-caf7226bcf26 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:23.954860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.956110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.956397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.957004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.964743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.966189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:23.977636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:23.978525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:23.979376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 250/514] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:23.979927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:23.985268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:23.985841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:23.986128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:23.986407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:23.987094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:23.987401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:23.989828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.001237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 265/515] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:24.002051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.009915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.085436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.087232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1627ab-731c-42d2-9a03-02adada2850b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.105665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c2c71-1d71-424c-b11b-01e0d739519a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.121078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.121691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.121977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.122254 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.122933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:24.123236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.124607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.135926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 251/516] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:24.136738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.142885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.143544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.143834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.144114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.144930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:24.145281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.146588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.158348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 266/517] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:24.159143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.221691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.238087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3011b3b-3813-49ee-96b2-33f1b1532508 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.261814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97793e3-9abe-43cb-b734-0aec7f70f7cb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.273540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.274164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.274451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.274730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.284771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.286745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.295397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.296110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.296580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.296777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.297569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:24.297876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.299288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:24.301987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:24.302826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:24.303651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:24.303651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:24.303651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:24.303651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:24.303651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 252/518] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:24.304316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.325475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.337878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 267/519] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:24.338929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.414017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47f39a-883c-4269-975b-a2634d11db16 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.438961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc093f9f-c883-480a-b049-573af8731a75 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.447897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.448558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.448844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.449124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.451732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.456513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.459275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:24.470999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:24.471836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:24.472730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 253/520] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:24.473937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.479433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.480062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.480341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.480646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.481322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.482765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.504379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 268/521] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:13:24 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:24.505029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.569828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.601610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9991f-bd28-4f32-8b4e-b120d157aa12 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.626515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9985a7-e841-465d-bc45-0d50d596961d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.638781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.639446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.639735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.640016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.649265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.650935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:24.663326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:24.664206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 254/522] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:24.665739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.672282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.673540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.673540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.673540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.674191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:24.674503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.676484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.690358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 269/523] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:24.691021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.740766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.788833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375ec16-5063-45fc-831b-d64ad9306db1 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.810145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4e098-20b5-47da-a179-ad3e68707e34 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:24.824423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.825080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.825361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.825642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.826209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.850964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:24.851596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:24.852021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:24.852300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:24.853543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:24.853973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.855862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.883289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 270/524] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:24.883289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:24.896797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:24.898271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:24.954188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:24.999428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281f35-21e9-42da-ac2b-c60f84e08bfe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.000086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 255/525] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:25.001406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.043436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.043984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.044749 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.045033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.045825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:25.046501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.048288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.067300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 271/526] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jul 29 06:13:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:25.070149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.103107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:25.109171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2bc7e-c14c-4041-9db0-59bfc7aeb8da None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.144040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.144746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.145640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.145965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.153479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.154931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:25.166618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:25.167473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:25.168291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 256/527] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:25.169459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.184903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c318f6-a90c-4f1e-9fe3-aea62cac7bf2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.189038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:25.222008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.222628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.223007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.223291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.224101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bbe64416672b408f8cdf93220cc31ef5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:25.224415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.225900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.240589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 272/528] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/role_assignments?user.id=bbe64416672b408f8cdf93220cc31ef5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:25.240589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.287254 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dbd89-7c97-4561-bfaf-6083d816a5cf None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.328143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.328759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.329052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.329335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.343002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.344617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.349271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:25.357745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca6ddb-4ae7-4e68-a4a1-e15fa1d1cf95 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:25.362784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:25.364381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:25.365320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:25.365320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:25.365320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:25.365320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:25.365320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 257/529] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:25.366354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.400861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.401895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.402381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.402859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.439169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:25.475072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55128f-3efd-48e4-8b3a-426eb928b627 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.480991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.482801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.510572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.511204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.511498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.511781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.512623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:25.512940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.514371 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.538806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 258/530] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:25.539718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.588861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:25.602320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 273/531] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:25.603696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.655458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db80f5-206b-4863-acfc-36f432f06f66 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.693476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:25.704023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.705162 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.706179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.706684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.708140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:25.708742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.711155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.729583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b8158-ae58-4988-922b-cda80200d10b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.734016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 259/532] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:25.735155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.775725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.776481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.776815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.777215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.778010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bc4b8f7545934151aafa986da871b944) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:25.778330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.780032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.792631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 274/533] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/role_assignments?user.id=bc4b8f7545934151aafa986da871b944&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:25.795715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:25.908229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd44712-f476-43fd-8aff-ca2c1878852a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:25.933674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:25.947915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:25.948949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:25.949381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:25.949807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:25.962984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:25.964735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:25.981530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:25.982589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:25.983637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:25.983637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:25.983637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:25.983637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:25.985266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 260/534] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:25.997058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:26.038959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f9733-535a-4f59-8846-3e7ce85d1198 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:26.073017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:26.102974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:26.103867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.104300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.104748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.122300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:26.123775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:26.139788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:26.141013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:26.142109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 275/535] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:26.148594 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:26.174254 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f24d5b-1da5-4b24-af45-78bf04c6801e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:26.242220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:26.243172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.244449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.247421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.248282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:26.248804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:26.250386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:26.277642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 261/536] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:26.281730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:26.358043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9f07e-7aae-4846-9465-b272427a0629 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:26.403713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:26.442372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:26.442561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:26.443111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.443450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.443702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.444483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:26.444855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:26.446352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff263f-4087-4813-b493-5cc94af95d85 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:26.452112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:26.482893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 276/537] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:26.485782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:26.486642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.487112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.487897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.489122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:26.496454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:26.499052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:26.513221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:26.514188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:26.515009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:26.517613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 262/538] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:26.522922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:26.628502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fb30e-3fe6-4a6d-ba29-c0dbe27671f1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:26.686441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31287d-56e5-4225-aee2-0611d8ce09c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:26.687061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31287d-56e5-4225-aee2-0611d8ce09c6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.687340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31287d-56e5-4225-aee2-0611d8ce09c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.687612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31287d-56e5-4225-aee2-0611d8ce09c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.688393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31287d-56e5-4225-aee2-0611d8ce09c6 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=ec1feaf015a847ffa719fd3a53de02e5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:26.688734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31287d-56e5-4225-aee2-0611d8ce09c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:26.690172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31287d-56e5-4225-aee2-0611d8ce09c6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:26.712746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 277/539] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:26 2022] GET /identity/v3/role_assignments?user.id=ec1feaf015a847ffa719fd3a53de02e5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:26.714676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1221109-6624-43c6-8434-8544c306b3bf None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.715075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1221109-6624-43c6-8434-8544c306b3bf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.715447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1221109-6624-43c6-8434-8544c306b3bf None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.717319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 278/540] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:26.723556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54579cb2-b803-41d6-bbc1-64542d6d362b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.723984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54579cb2-b803-41d6-bbc1-64542d6d362b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.724453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54579cb2-b803-41d6-bbc1-64542d6d362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.752298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-54579cb2-b803-41d6-bbc1-64542d6d362b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:26.753941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-54579cb2-b803-41d6-bbc1-64542d6d362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:26.803577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:26.824074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978ba1c-8ab6-4b55-afd0-83fce097772c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:26.837311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 279/541] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:26.861352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54579cb2-b803-41d6-bbc1-64542d6d362b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:26.919499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:26.920426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:26.920941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:26.921357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:26.922359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:26.922807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:26.924453 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:26.945955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 263/542] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:26.951655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:26.983236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54579cb2-b803-41d6-bbc1-64542d6d362b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:27.007755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:27.023462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:27.024131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.024424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.024754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.037305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:27.042733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:27.044118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:27.044950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 280/543] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:27.052927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:27.099612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:27.136748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dfaec-1027-444f-8cb7-e54f9a284c34 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:27.187153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8063fa-96b7-4519-8978-5f3559cef42f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:27.216943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:27.217569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.217849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.218122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.218905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=edf0a00538cf4e10b3bf266f48ea8cb9) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:27.219213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:27.220661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:27.234967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 264/544] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:26 2022] GET /identity/v3/role_assignments?user.id=edf0a00538cf4e10b3bf266f48ea8cb9&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:27.245334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d7550-3f03-42ff-b9e0-8d9ab024e3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:27.245979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7550-3f03-42ff-b9e0-8d9ab024e3cc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.246262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7550-3f03-42ff-b9e0-8d9ab024e3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.246537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7550-3f03-42ff-b9e0-8d9ab024e3cc None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.247273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d7550-3f03-42ff-b9e0-8d9ab024e3cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:27.247576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d7550-3f03-42ff-b9e0-8d9ab024e3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:27.249052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d7550-3f03-42ff-b9e0-8d9ab024e3cc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:27.258656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 281/545] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:27.264555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:27.372577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:27.389608 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12468c1d-4c1f-4a4e-9e46-bec8226d40a4 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:27.429195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09230f02-c621-4646-b233-bd7523a3eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:27.429870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09230f02-c621-4646-b233-bd7523a3eb98 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.430158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09230f02-c621-4646-b233-bd7523a3eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.430436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09230f02-c621-4646-b233-bd7523a3eb98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.442941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09230f02-c621-4646-b233-bd7523a3eb98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:27.444420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09230f02-c621-4646-b233-bd7523a3eb98 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:27.460430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a11a9-99b2-44e1-9f08-a131bf675554 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.460757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a11a9-99b2-44e1-9f08-a131bf675554 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.461086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a11a9-99b2-44e1-9f08-a131bf675554 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-09230f02-c621-4646-b233-bd7523a3eb98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:27.463446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:27.464283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:27.465133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:27.465133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:27.465133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:27.465133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:27.465133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 265/547] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:27.465399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 282/547] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:27.470135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de35be-7837-4f9b-8359-43e341ccd5e2 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.470440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de35be-7837-4f9b-8359-43e341ccd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.470727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de35be-7837-4f9b-8359-43e341ccd5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.471246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a11a9-99b2-44e1-9f08-a131bf675554 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:27.494742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a3de35be-7837-4f9b-8359-43e341ccd5e2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:27.495879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a3de35be-7837-4f9b-8359-43e341ccd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:27.594006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 266/548] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:27.618514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:27.631278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a11a9-99b2-44e1-9f08-a131bf675554 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:27.656837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de35be-7837-4f9b-8359-43e341ccd5e2 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:27.693287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:27.693926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.694250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.694539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.695238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:27.695550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:27.696979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:27.719147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 283/549] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:27.724993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:27.774074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:27.836853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de35be-7837-4f9b-8359-43e341ccd5e2 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:27.897054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:27.897717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:27.898006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:27.898289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:27.906385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d8137-2243-4cbe-9b4f-b50c7aaa67e4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:27.920665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:27.925582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:27.927856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:27.928846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:27.934904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 267/550] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:27.936422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.007864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.008514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.008819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.009095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.021786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.025313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:28.045320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:28.047063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:28.048201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 284/551] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:28.057607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.077647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cab238-1572-44fb-bd54-553ebd19776d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.089965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.107916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.118647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.119533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.119824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.120106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.120888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:28.121171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.122542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.133764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 268/552] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:28.141990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.254953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da02077-515a-4337-923b-69ce6dae8d11 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.264450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab56cb-ca85-4b9c-8ffd-3f86b264ef24 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.295318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a247e-2f36-4437-b896-9163818f04f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.295874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a247e-2f36-4437-b896-9163818f04f1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.296159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a247e-2f36-4437-b896-9163818f04f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.296438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a247e-2f36-4437-b896-9163818f04f1 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.297358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a247e-2f36-4437-b896-9163818f04f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:28.297674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a247e-2f36-4437-b896-9163818f04f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.299230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a247e-2f36-4437-b896-9163818f04f1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.306617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b8f8e-3985-4fbd-838c-947af0bdbb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.307269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8f8e-3985-4fbd-838c-947af0bdbb09 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.307555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8f8e-3985-4fbd-838c-947af0bdbb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.307833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8f8e-3985-4fbd-838c-947af0bdbb09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.319851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b8f8e-3985-4fbd-838c-947af0bdbb09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.321354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b8f8e-3985-4fbd-838c-947af0bdbb09 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.324496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 285/553] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:28.325929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a594f6-3362-4adb-a338-e50e90165a2b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.326226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a594f6-3362-4adb-a338-e50e90165a2b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.326606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a594f6-3362-4adb-a338-e50e90165a2b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.328094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 286/554] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:28.328824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a594f6-3362-4adb-a338-e50e90165a2b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-2e6b8f8e-3985-4fbd-838c-947af0bdbb09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:28.339762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:28.340696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:28.341947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:28.341947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:28.341947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:28.341947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:28.341947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 269/555] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:28.342496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b620-ff0b-4bd6-bffe-a2474c8117a8 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.343690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b620-ff0b-4bd6-bffe-a2474c8117a8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.343690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b620-ff0b-4bd6-bffe-a2474c8117a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.362402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e172b620-ff0b-4bd6-bffe-a2474c8117a8 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:28.364176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e172b620-ff0b-4bd6-bffe-a2474c8117a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:28.401934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.417144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.441309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 270/556] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:28.442492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b620-ff0b-4bd6-bffe-a2474c8117a8 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.453163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a594f6-3362-4adb-a338-e50e90165a2b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.505156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.505796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.506423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.506567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.552781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b620-ff0b-4bd6-bffe-a2474c8117a8 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.580814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.583385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.591467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.592346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.592902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.593364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.594354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:28.594810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.596565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.610160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 271/557] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:28.611422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.645431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.666891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.701402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 287/558] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:28.702337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.728885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901da2-4d50-4764-8367-949602798bb4 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.767642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.768404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.768727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.769010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.781247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:28.787366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:28.788780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 272/559] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:28.789586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.817525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdb27e-29dc-409f-9a77-06765734e892 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.855507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.856152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.856430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.856776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.864977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.866550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:28.879783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:28.882147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 288/560] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:28.884429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.889519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.900979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef923b1-b34e-4247-acdf-9ab82c5ea073 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:28.922009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:28.940767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:28.941422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:28.941741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:28.942017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:28.954041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:28.955373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:28.971703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:28.973407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:28.974415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:28.974415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:28.974415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:28.974415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:28.978245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 273/561] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:28.979513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:28.998997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a1cd8-c4b2-42ef-bbf7-2706ac15bd62 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.037870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.038850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.039307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.039780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.040718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:29.041173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.042900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.058364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 289/562] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:29.060014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.091209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165b5fe-f4cb-4210-876b-4a1503300c84 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.127891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.128181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.128462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.129984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:29.130303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.131735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.146030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.150459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 274/563] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:13:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:29.151271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.165924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.172703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8017f2-9157-4aac-b22d-472fd7b84091 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.209700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d9b6e-ac2f-4d69-bc51-bc19cd4e9896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.210342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9b6e-ac2f-4d69-bc51-bc19cd4e9896 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.210637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9b6e-ac2f-4d69-bc51-bc19cd4e9896 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.210899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9b6e-ac2f-4d69-bc51-bc19cd4e9896 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.211584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d9b6e-ac2f-4d69-bc51-bc19cd4e9896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:29.211892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d9b6e-ac2f-4d69-bc51-bc19cd4e9896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.213328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d9b6e-ac2f-4d69-bc51-bc19cd4e9896 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.226158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 290/564] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:29.227395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab862a-31e3-4426-afb0-619962bf61c5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.227692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab862a-31e3-4426-afb0-619962bf61c5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.227973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab862a-31e3-4426-afb0-619962bf61c5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.229227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 291/565] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:29.229922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab862a-31e3-4426-afb0-619962bf61c5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.267816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41462ce-5908-447a-a74d-d4fd6e8db3ad None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.302322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668006b5-261d-4b6d-a392-01ed16bad229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.302977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668006b5-261d-4b6d-a392-01ed16bad229 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.303261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668006b5-261d-4b6d-a392-01ed16bad229 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.303538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668006b5-261d-4b6d-a392-01ed16bad229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.316032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668006b5-261d-4b6d-a392-01ed16bad229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.318218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668006b5-261d-4b6d-a392-01ed16bad229 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-668006b5-261d-4b6d-a392-01ed16bad229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:29.333523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:29.334412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:29.335326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:29.335326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:29.335326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:29.335326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:29.335326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 275/566] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:29.336098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a6a67-877d-4945-8c14-9b5abb7b69f4 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.336196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a6a67-877d-4945-8c14-9b5abb7b69f4 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.336380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a6a67-877d-4945-8c14-9b5abb7b69f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.343013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab862a-31e3-4426-afb0-619962bf61c5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.354684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ff0a6a67-877d-4945-8c14-9b5abb7b69f4 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:29.355967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ff0a6a67-877d-4945-8c14-9b5abb7b69f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:29.380459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.381122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.381408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.381684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.393335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.395493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:29.410761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:29.411672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:29.412579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:29.412579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:29.412579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:29.412579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:29.414323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.416930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 292/567] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:29.417768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.428887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.437843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 276/568] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:29.438577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a6a67-877d-4945-8c14-9b5abb7b69f4 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.541766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7d88d-0f7c-4f74-9fdd-4e57d9329c6d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.551597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a6a67-877d-4945-8c14-9b5abb7b69f4 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.578814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.579462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.579800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.580084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.587250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.588723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.588850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.589348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.589629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.589904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.590586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:29.590889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.592230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:29.601514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:29.602413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:29.603285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 293/569] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:29.603801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.614502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 277/570] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:29.614502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.656564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.673377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.709298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4020a-b71a-4324-8243-e50fdf1d5544 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.715106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002dcd-84b8-4d10-b2ac-6bfdd56294c0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.743550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.744179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.744463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.744772 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.745461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:29.745776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.747156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.753176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.754293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.754293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.754504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.758808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 294/571] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:29.759596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.769145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:29.775710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:29.776641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:29.777477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 278/572] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:29.778155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.866869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c1767-8e3b-4f33-ae0b-a21662d2f529 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.885706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f2e7f-3451-4f64-95c3-f94c3cdd9b95 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:29.887187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.893310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:29.902441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0f474-04a9-4ceb-99ba-d5546bde462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.903068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0f474-04a9-4ceb-99ba-d5546bde462a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.903352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0f474-04a9-4ceb-99ba-d5546bde462a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.903631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0f474-04a9-4ceb-99ba-d5546bde462a None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.904370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0f474-04a9-4ceb-99ba-d5546bde462a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:29.904710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0f474-04a9-4ceb-99ba-d5546bde462a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.906132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0f474-04a9-4ceb-99ba-d5546bde462a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.917804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 295/573] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:29.918988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c050ffd-eef0-48cd-82a2-a76db33d3fd0 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.919278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c050ffd-eef0-48cd-82a2-a76db33d3fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.919557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c050ffd-eef0-48cd-82a2-a76db33d3fd0 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.920378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:29.920929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 296/574] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:29.921052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:29.921323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:29.921481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c050ffd-eef0-48cd-82a2-a76db33d3fd0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:29.921658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] PATH_INFO: `/v3/users/glance` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:29.930020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:29.931456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:29.943184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:29.944147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:29.945109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 279/575] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:29.945672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.039269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c050ffd-eef0-48cd-82a2-a76db33d3fd0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.043059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dcf53-56fd-4361-bc95-80cb5a9d48da None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.073893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af3785-0ad8-4f4c-a39d-b1a42e68b946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.074541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3785-0ad8-4f4c-a39d-b1a42e68b946 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.074828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3785-0ad8-4f4c-a39d-b1a42e68b946 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.075107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3785-0ad8-4f4c-a39d-b1a42e68b946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.077489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.078111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.078398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.078680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.079390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:30.079699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.081357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.082133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af3785-0ad8-4f4c-a39d-b1a42e68b946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.083631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af3785-0ad8-4f4c-a39d-b1a42e68b946 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.089347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 280/576] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:30.090138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f9af3785-0ad8-4f4c-a39d-b1a42e68b946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:30.095208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:30.096123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:30.097372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 297/577] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:30.098407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34522c-2c85-4235-a5fc-1ac6a83fa203 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.098407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34522c-2c85-4235-a5fc-1ac6a83fa203 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.098611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34522c-2c85-4235-a5fc-1ac6a83fa203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.118087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.118728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dc34522c-2c85-4235-a5fc-1ac6a83fa203 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:30.120096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dc34522c-2c85-4235-a5fc-1ac6a83fa203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:30.164005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.188325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 298/578] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:30.189355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34522c-2c85-4235-a5fc-1ac6a83fa203 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.194736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce71c2b-1e08-48a7-b84b-f2dbdae41c8c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.228333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.228985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.229275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.229556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.230337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=e35b790a4a5b4306a5537984d0b449e6) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:30.230652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.232073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.237398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 281/579] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/role_assignments?user.id=e35b790a4a5b4306a5537984d0b449e6&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:30.238163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.311082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34522c-2c85-4235-a5fc-1ac6a83fa203 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.337044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.351748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.352404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.352730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.353017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.353824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:30.354139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.354735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4d79-03ad-4aa0-9876-e2e68726cdaf None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.355637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.379646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 299/580] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:30.380652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.394583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.395805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.396092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.396682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.407361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.409216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.420433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:30.423960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:30.424868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:30.426519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:30.426519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:30.426519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:30.426519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:30.430529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 282/581] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:30.431313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.493273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fdf92-57fd-4d19-b983-9a6bfdc0121f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.529340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.529969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.530259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.530542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.531290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:30.531605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.533068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.543616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145b60b-beab-4c5a-b460-b642e0ba4671 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.546527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 300/582] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:30.547039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.585428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.586081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.586374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.586657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.594762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.598278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:30.604836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:30.605807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:30.606643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 283/583] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:30.607349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.657486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.665907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f7276-b9a2-4a01-990a-e5a14fd43aa5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.703712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.704477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.704810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.705183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.711734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa73351-0dfa-4ca3-9841-1ce1f1bfc6ff None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.719670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.719670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:30.737991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:30.739191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:30.740123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:30.740123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:30.740123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:30.740123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:30.740123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 301/584] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:30.741055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.759028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.759824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.760188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.760591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.761266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:30.761579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.762945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.775519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 284/585] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:30.776330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.841541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.852549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695a4dd-6f5a-4a68-a576-415b16c7ec46 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.886017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56470851-e0e4-40d4-94f5-a3d758dbb28b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:30.888229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.888963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.889248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.889539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/bbe64416672b408f8cdf93220cc31ef5/roles/b8e60301309b44f4acfea6244f79788b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.904742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=bbe64416672b408f8cdf93220cc31ef5, role_id=b8e60301309b44f4acfea6244f79788b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.906183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.914809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:30.932565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 302/586] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:30 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/bbe64416672b408f8cdf93220cc31ef5/roles/b8e60301309b44f4acfea6244f79788b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:30.932565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:30.932903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:30.932903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:30.932903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:30.932903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:30.933242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:30.933242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:30.933242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:30.937772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 285/587] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:30.938720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.076370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef55182-644d-41f4-ba76-5b6f5cdc19fe None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.084263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9d6cc-ec65-4095-8456-99b61a5302a5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.110412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:31.128375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.130018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.130401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.130788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.132524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.133180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.133465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.134199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.134438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:31.134750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.136138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.139898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.141364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.149497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 303/588] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:31.150478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:31.160262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:31.161365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:31.162281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 286/589] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:31.162855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.236510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:31.282794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196402-8cd6-436b-b62a-ad4b9dfc8c0b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.298415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e3218-d397-417c-9ffc-e178092fcd0e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.320195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.321081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.321619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.322001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.335799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.336941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.337607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.337789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.337897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.338152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.345823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.347259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:31.354249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:31.356254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:31.358457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:31.358457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:31.358457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:31.358457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:31.364561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 304/590] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:31.364793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:31.366693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:31.368875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:31.369881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.370037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 287/591] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:31.370037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.414761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:31.471547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be562f7-0c1e-4d68-8937-4401e7a4c14b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.476666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f8576-e2ba-4af3-b0a9-2db537a5a174 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.482525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:31.510326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.510970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.511397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.511760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.512565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:31.512888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.513849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.514477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.514649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.514932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.515281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.516041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:31.516443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.517907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.530870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 288/592] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:31.531939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.539764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 305/593] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:31.540679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.637910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b9d3-bc49-4f26-8adf-03cecbe279ac None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.639569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:31.652328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d894e0-645d-44ed-92ad-d945a072f3a7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.677376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.678010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.678298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.678582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.679331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:31.679645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.681098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.693054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.693707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.693988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.694264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.698587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 289/594] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:31.699426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.708585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.708585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:31.739636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:31.740621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 06:13:31.741570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:31.742938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 306/595] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:31.744066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.781048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:31.845692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50994474-e81b-44dc-9339-c8ec1d2ae4b7 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.876119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d579d8e-24f9-42c2-8e94-c72632e76b7d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:31.893605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.895756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.896276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.896807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.912911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:31.913587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:31.913870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:31.914147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/bc4b8f7545934151aafa986da871b944/roles/b8e60301309b44f4acfea6244f79788b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:31.914422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.916348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.930352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=bc4b8f7545934151aafa986da871b944, role_id=b8e60301309b44f4acfea6244f79788b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:31.931890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:31.936359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:31.937528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:31.938739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:31.938739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:31.938739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:31.938739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:31.938739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 290/596] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:31.939118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.952687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 307/597] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:31 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/bc4b8f7545934151aafa986da871b944/roles/b8e60301309b44f4acfea6244f79788b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:31.953741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:31.968960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.136927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.153588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b533cf-ccd6-4c5c-bc6f-e3062e0d1893 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.175081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27a46-735e-4180-b772-344b05641a5c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.193967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.194693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.194991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.195265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.196064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:32.196376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.198539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:32.222456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 291/598] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:32.224866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdab583-8db2-4881-ac44-ee3251092ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.225503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab583-8db2-4881-ac44-ee3251092ceb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.225789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab583-8db2-4881-ac44-ee3251092ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.226068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab583-8db2-4881-ac44-ee3251092ceb None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.226765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdab583-8db2-4881-ac44-ee3251092ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:32.227071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdab583-8db2-4881-ac44-ee3251092ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.228456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdab583-8db2-4881-ac44-ee3251092ceb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:32.233539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.242797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 308/599] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:32.244165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cf9e8-213e-49f5-bd7f-6fd0aff07d64 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.244462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cf9e8-213e-49f5-bd7f-6fd0aff07d64 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.244778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cf9e8-213e-49f5-bd7f-6fd0aff07d64 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.246947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 309/600] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:32.247732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cf9e8-213e-49f5-bd7f-6fd0aff07d64 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.327492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.365697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b90a-0e98-49db-9bb8-07ac12351e8b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.383642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cf9e8-213e-49f5-bd7f-6fd0aff07d64 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.403566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de525a2-c1e3-4930-b390-53067eec8390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.404195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de525a2-c1e3-4930-b390-53067eec8390 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.404475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de525a2-c1e3-4930-b390-53067eec8390 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.404848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de525a2-c1e3-4930-b390-53067eec8390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.416353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de525a2-c1e3-4930-b390-53067eec8390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.417874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de525a2-c1e3-4930-b390-53067eec8390 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:32.431102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.432046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.432590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.433066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-8de525a2-c1e3-4930-b390-53067eec8390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:32.435731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:32.436625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:32.437499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:32.437499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:32.437499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:32.437499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:32.437499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 292/601] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:32.438395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f663e-6612-407c-9016-e4ab17f36d1c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.438803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f663e-6612-407c-9016-e4ab17f36d1c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.439242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f663e-6612-407c-9016-e4ab17f36d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.444504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.446200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:32.449289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.460518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ca2f663e-6612-407c-9016-e4ab17f36d1c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:32.461792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ca2f663e-6612-407c-9016-e4ab17f36d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:32.466744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:32.467623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:32.468505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:32.476232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 310/602] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:32.476232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.560973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 293/603] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:32.561582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f663e-6612-407c-9016-e4ab17f36d1c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.595591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ca2ca-3a45-4621-a8bc-61694c9fa1a8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.630845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.631478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.631761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.632103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.633055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:32.633553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.634719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.635976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:32.649944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 311/604] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:32.650851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.684013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f663e-6612-407c-9016-e4ab17f36d1c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.712036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.724198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.725227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.725811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.726299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.727288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:32.727804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.729463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:32.748638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 294/605] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:32.749871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.762639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d763e14-0983-47c7-ad17-880ae1d1403b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.799250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.800046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.800285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.800567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.812149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:32.818298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:32.819220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:32.820062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 312/606] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:32.826423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.872696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d31b0f-16de-4e1e-a6b2-303904369bf0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.888804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.912657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.913699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.914195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.914734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.922334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.924208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:32.930151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638ff6c-deaf-40ba-820d-d8ee2ab75af7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:32.937244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:32.938104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:32.938986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:32.942230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 295/607] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:32.943377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:32.950720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:32.969490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:32.970170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:32.970462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:32.970738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/ec1feaf015a847ffa719fd3a53de02e5/roles/b8e60301309b44f4acfea6244f79788b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:32.985605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=ec1feaf015a847ffa719fd3a53de02e5, role_id=b8e60301309b44f4acfea6244f79788b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:32.987056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.012563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 313/608] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:32 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/ec1feaf015a847ffa719fd3a53de02e5/roles/b8e60301309b44f4acfea6244f79788b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:33.012563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.116998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80843cfe-b56d-4e7b-8f98-93225e46d5c2 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.165191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7348bf-ce9e-4c80-8ede-afef6df1f3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.165937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7348bf-ce9e-4c80-8ede-afef6df1f3fb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.166220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7348bf-ce9e-4c80-8ede-afef6df1f3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.166496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7348bf-ce9e-4c80-8ede-afef6df1f3fb None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.167215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7348bf-ce9e-4c80-8ede-afef6df1f3fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:33.167728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7348bf-ce9e-4c80-8ede-afef6df1f3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.169061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef1185-4e44-4aa9-9ec2-20187f88b9d2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.169511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7348bf-ce9e-4c80-8ede-afef6df1f3fb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.179861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 296/609] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:33.181299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc9712-d533-4afd-83f6-51d20e4673f7 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.181585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc9712-d533-4afd-83f6-51d20e4673f7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.181860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc9712-d533-4afd-83f6-51d20e4673f7 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.187367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 297/610] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:33.188156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc9712-d533-4afd-83f6-51d20e4673f7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.208726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec840e-6427-45af-91be-694ec56e7b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.209472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec840e-6427-45af-91be-694ec56e7b95 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.209825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec840e-6427-45af-91be-694ec56e7b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.210218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec840e-6427-45af-91be-694ec56e7b95 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.211228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec840e-6427-45af-91be-694ec56e7b95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:33.211609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec840e-6427-45af-91be-694ec56e7b95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.213433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec840e-6427-45af-91be-694ec56e7b95 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.229253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 314/611] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:33.230910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4504-6ab6-4c93-87e6-6594c6943a0a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.231270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4504-6ab6-4c93-87e6-6594c6943a0a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.231624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4504-6ab6-4c93-87e6-6594c6943a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.233519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:33.251935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-62dd4504-6ab6-4c93-87e6-6594c6943a0a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:33.253389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-62dd4504-6ab6-4c93-87e6-6594c6943a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:33.283116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:33.308902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc9712-d533-4afd-83f6-51d20e4673f7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.333183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 315/612] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:33.334310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd4504-6ab6-4c93-87e6-6594c6943a0a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.362274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.362932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.363266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.363545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.379418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.380964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:33.397777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:33.398679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:33.399589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:33.399589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:33.399589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:33.399589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:33.399589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 298/613] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:33.400305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.449212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd4504-6ab6-4c93-87e6-6594c6943a0a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.487292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.489059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.489592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.490075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/edf0a00538cf4e10b3bf266f48ea8cb9/roles/b8e60301309b44f4acfea6244f79788b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.498779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:33.509144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=edf0a00538cf4e10b3bf266f48ea8cb9, role_id=b8e60301309b44f4acfea6244f79788b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.511068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.512462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028d0f4-b962-4164-bad6-f25f782bf886 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.527194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 316/614] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:33 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/edf0a00538cf4e10b3bf266f48ea8cb9/roles/b8e60301309b44f4acfea6244f79788b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:33.528508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.542006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:33.551437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.552241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.552581 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.552865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.566169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:33.572862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:33.573796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:33.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 299/615] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:33.580401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.657709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86c15a-104c-415b-bc51-51099a733704 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.692617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57a59-f81b-4238-b34d-2ca2ce9c765e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.698961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.700168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.700700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.701192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.702124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:33.702743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.704512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.718630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 317/616] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:33.723771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.732495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.733223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.733502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.733775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.734538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:33.734843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.736301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.746856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 300/617] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:33.752613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.766711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:33.807332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:33.849115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cf31b-a64d-4ec7-b40a-d4c7b128b087 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.864881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe523e73-e1a0-47ad-96dc-82f3674cfdd9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:33.887669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.888841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.889329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.889805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.899753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.901322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.902965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:33.904700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:33.905212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:33.905691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:33.922354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:33.923346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:33.924275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:33.931207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 318/618] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:33.932293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:33.934144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:33.935729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:33.956240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:33.957733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:33.958763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:33.958763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:33.958763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:33.958763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:33.961006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 301/619] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:33.966210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.038853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:34.103936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eef7d1-3048-41cf-8276-89bc6727cb3b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.150708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c2872-b9fa-4593-978a-e27c30e86b65 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.180357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:34.181836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.182474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.182756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.183032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.183721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:34.184121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:34.185424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:34.193066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.194036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.194547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.195041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.196027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:34.196573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:34.198207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:34.215146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 302/620] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:34.219970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.221721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 319/621] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 29 06:13:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:34.229726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.364346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28a13-5454-4412-81fa-afc83738b0c0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.400380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bfbff-a7cb-44d2-9d81-67c37bf0359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.401061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bfbff-a7cb-44d2-9d81-67c37bf0359f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.401348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bfbff-a7cb-44d2-9d81-67c37bf0359f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.401627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bfbff-a7cb-44d2-9d81-67c37bf0359f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.411319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab2100-bf11-4b29-9086-8c10310dce73 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.413429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bfbff-a7cb-44d2-9d81-67c37bf0359f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:34.415174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bfbff-a7cb-44d2-9d81-67c37bf0359f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-bd8bfbff-a7cb-44d2-9d81-67c37bf0359f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:34.431305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:34.433297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 320/622] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fd9f2-b00f-4bd1-9ce3-d84df38b76f2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fd9f2-b00f-4bd1-9ce3-d84df38b76f2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.435076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fd9f2-b00f-4bd1-9ce3-d84df38b76f2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.436491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 321/623] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:34.445027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fd9f2-b00f-4bd1-9ce3-d84df38b76f2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.459657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:34.482513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc0c78-c3d0-4f30-b279-60304cc71819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.484040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:34.485172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc0c78-c3d0-4f30-b279-60304cc71819 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.485172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc0c78-c3d0-4f30-b279-60304cc71819 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.485446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc0c78-c3d0-4f30-b279-60304cc71819 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.498037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc0c78-c3d0-4f30-b279-60304cc71819 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:34.499680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc0c78-c3d0-4f30-b279-60304cc71819 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-7cbc0c78-c3d0-4f30-b279-60304cc71819 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:34.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:34.517115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:34.518066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:34.522343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 303/624] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:34.524317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8171e0-744e-4fbb-a60d-0a75d0761ea9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.524651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8171e0-744e-4fbb-a60d-0a75d0761ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.524939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8171e0-744e-4fbb-a60d-0a75d0761ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.547946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3a8171e0-744e-4fbb-a60d-0a75d0761ea9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:34.549526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3a8171e0-744e-4fbb-a60d-0a75d0761ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:34.581105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fd9f2-b00f-4bd1-9ce3-d84df38b76f2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.630668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.631863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.632332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.632971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.634091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:34.634581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:34.635542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 304/625] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:34.636414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:34.636651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8171e0-744e-4fbb-a60d-0a75d0761ea9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.650682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 322/626] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:34.661595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.753555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8171e0-744e-4fbb-a60d-0a75d0761ea9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.769909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:34.775720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:34.780282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7e6f2-8f90-418f-b792-67684dc1b592 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.797061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.797701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.797985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.798302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.799048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:34.799354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:34.801955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:34.819700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.820731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.825597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 305/627] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:34.825597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.825949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.826475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.834519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:34.836288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:34.848596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:34.849658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:34.850568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:34.853160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 323/628] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:34.854150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:34.934410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b5e4d-8578-4bf5-bcb3-40dc9f9de74e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.960152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae558c-6935-4f6d-a020-13da4f20f84c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:34.977428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:34.978104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:34.978392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:34.978668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:34.990156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:34.995470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:34.996375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:34.997242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 306/629] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:34.997936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.003991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.005290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.005839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.006328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.031421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.043975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.102558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.104979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.109288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e4771-7e6b-404a-b319-49fe23e22d75 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.148644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.150515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.151051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.151536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.152494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:35.153036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.155224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.173747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 307/630] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:35.175512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.228956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 324/631] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:35.229849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.291164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.292003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.304370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18096411-3e0c-4631-b0b9-eb827d619df8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.336316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049d450-8275-48f6-9d49-c0bd99ca9daf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.341435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.342329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.342820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.343303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.344233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:35.344790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.346374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.356275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 308/632] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:35.357363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.374476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.375341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.375462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.375706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/e35b790a4a5b4306a5537984d0b449e6/roles/b8e60301309b44f4acfea6244f79788b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.391508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=e35b790a4a5b4306a5537984d0b449e6, role_id=b8e60301309b44f4acfea6244f79788b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.392974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.409179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 325/633] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:35 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/e35b790a4a5b4306a5537984d0b449e6/roles/b8e60301309b44f4acfea6244f79788b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:35.410770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.474623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1240a65-1d53-45b9-aa73-fb185d9fc8e6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.514964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1197e9-7425-4aa9-8c7e-417e3757aaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.516017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1197e9-7425-4aa9-8c7e-417e3757aaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.516510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1197e9-7425-4aa9-8c7e-417e3757aaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.517031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1197e9-7425-4aa9-8c7e-417e3757aaf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.529732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1197e9-7425-4aa9-8c7e-417e3757aaf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.531644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1197e9-7425-4aa9-8c7e-417e3757aaf6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.532622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b81521-71d0-4d27-9c56-0de7d3de3cf8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-de1197e9-7425-4aa9-8c7e-417e3757aaf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:35.549264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:35.550181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:35.551461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:35.551461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:35.551461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:35.551461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:35.551693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.553977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 309/634] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:35.557413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbd0a4-3c2f-4c1f-b8af-a2592b436b93 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.559098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbd0a4-3c2f-4c1f-b8af-a2592b436b93 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.559609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbd0a4-3c2f-4c1f-b8af-a2592b436b93 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.561610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 310/635] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:35.562998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbd0a4-3c2f-4c1f-b8af-a2592b436b93 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.567985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.586183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634202e0-016a-42e9-9352-e82c2bb7c320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.586599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634202e0-016a-42e9-9352-e82c2bb7c320 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.586888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634202e0-016a-42e9-9352-e82c2bb7c320 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.587168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634202e0-016a-42e9-9352-e82c2bb7c320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.598681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634202e0-016a-42e9-9352-e82c2bb7c320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.599913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634202e0-016a-42e9-9352-e82c2bb7c320 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-634202e0-016a-42e9-9352-e82c2bb7c320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:35.618570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:35.619556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:35.620443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:35.620443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:35.620443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:35.620443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:35.620443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 326/636] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:35.621399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d780-3972-4bf8-b9ea-5250a80ffd3c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.621615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d780-3972-4bf8-b9ea-5250a80ffd3c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.621898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d780-3972-4bf8-b9ea-5250a80ffd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.641109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e290d780-3972-4bf8-b9ea-5250a80ffd3c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:35.642292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e290d780-3972-4bf8-b9ea-5250a80ffd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:35.693712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbd0a4-3c2f-4c1f-b8af-a2592b436b93 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.712205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 327/637] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:35.713096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290d780-3972-4bf8-b9ea-5250a80ffd3c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.742434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.743459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.744057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.744632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.745635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:35.746194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.748008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.762220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 311/638] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:35.763433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.805841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.815555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290d780-3972-4bf8-b9ea-5250a80ffd3c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.838610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:35.851479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.852108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.852394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.852705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.853394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:35.853695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:35.855052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:35.867759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 328/639] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:35.868617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.887243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e9984-bebe-4120-9d16-1e4a75d0cdb1 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:35.925398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:35.926981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:35.927470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:35.928049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:35.942216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:35.948448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:35.949516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:35.950356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:35.953055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 312/640] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:35.954190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:35.970694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06be49-87ff-43ec-8267-32be6c86b6d1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.004783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.005421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.005710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.005981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.012689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.014125 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:36.025998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:36.027142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 329/641] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:36.028273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.042672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.068616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60593587-9fad-4695-8b68-32dd2b574560 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.095919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.109140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7357fb7-91b9-42b9-845b-ed122c99cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.110141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357fb7-91b9-42b9-845b-ed122c99cb71 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.110589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357fb7-91b9-42b9-845b-ed122c99cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.110870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357fb7-91b9-42b9-845b-ed122c99cb71 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.119809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7357fb7-91b9-42b9-845b-ed122c99cb71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.121857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7357fb7-91b9-42b9-845b-ed122c99cb71 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-d7357fb7-91b9-42b9-845b-ed122c99cb71 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:36.135075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:36.136229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 313/642] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed60eb-286d-46c6-8834-4cffb559fb41 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.142545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed60eb-286d-46c6-8834-4cffb559fb41 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.143389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed60eb-286d-46c6-8834-4cffb559fb41 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.143389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 314/643] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:36.143389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed60eb-286d-46c6-8834-4cffb559fb41 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.157501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ee074-139c-4601-b00c-2e77b30ae2a9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.195367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.196292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.196672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.197068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.197885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:36.198188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.199660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.209442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 330/644] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:36.209442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.260308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed60eb-286d-46c6-8834-4cffb559fb41 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.289482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.299643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ab5c5-1d3d-4622-96e9-0992d8cef88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.300306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ab5c5-1d3d-4622-96e9-0992d8cef88b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.300639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ab5c5-1d3d-4622-96e9-0992d8cef88b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.300924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ab5c5-1d3d-4622-96e9-0992d8cef88b None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.301862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ab5c5-1d3d-4622-96e9-0992d8cef88b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:36.302223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ab5c5-1d3d-4622-96e9-0992d8cef88b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.303637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ab5c5-1d3d-4622-96e9-0992d8cef88b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.316626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 315/645] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:36.318579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9b8a-8f17-4779-bfcb-954504c5b101 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.318918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9b8a-8f17-4779-bfcb-954504c5b101 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.319145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9b8a-8f17-4779-bfcb-954504c5b101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.327515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef5d74-a536-493f-a79a-ec6224f7cc87 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.337888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-fafc9b8a-8f17-4779-bfcb-954504c5b101 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:36.339181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fafc9b8a-8f17-4779-bfcb-954504c5b101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:36.363423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.364158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.364437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.364736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.365517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:36.365822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.367329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.382484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.392246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 331/646] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:36.392246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.416671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 316/647] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:36.417545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9b8a-8f17-4779-bfcb-954504c5b101 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.509033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62b17-c300-4e2a-84f3-a59572e892f6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.524800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9b8a-8f17-4779-bfcb-954504c5b101 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.540770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.548803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.550806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.551484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.551925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.562562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.562713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.563208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.563493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.563784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.564003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.564598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bbe64416672b408f8cdf93220cc31ef5) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:36.564888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.566608 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.572829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 317/648] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/role_assignments?user.id=bbe64416672b408f8cdf93220cc31ef5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:36.573551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:36.589750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:36.590738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:36.591637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:36.591637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:36.591637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:36.591637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:36.596295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 332/649] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:36.598330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.638957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.729143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae235ea1-19c3-46a0-a08c-d3b67dcec981 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.743908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e0057-7268-43f7-bcc9-5cfeb22db53e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.776319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.777123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.777415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.777692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.789719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.790441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.790769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.791066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.802742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:36.807895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:36.808943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:36.810522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:36.812650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 333/650] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:36.813440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.868018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.870587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.872647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:36.919661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:36.948116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6d947-7ecf-4ed6-a3aa-0333a687c0cc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:36.991468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 318/651] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:36.992359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:36.993179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:36.993604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:36.994024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:36.994897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:36.995340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:36.996923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:36.997343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.010969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 334/652] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:37.016210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.105628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05546087-43cc-4f61-b48d-2c7027b17252 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.125647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b657d-1c06-4900-b5e9-b65bacb8c93c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.127415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.143893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.145342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.145726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.146226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.147042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:37.147573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.149066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.155012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.166038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 319/653] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:37.166038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.171458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.172316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.172793 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.173221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.184811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.186649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:37.203953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:37.204958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:37.206185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:37.206185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:37.206185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:37.206185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:37.209103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 335/654] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:37.210151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.277268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2442-0bcc-4557-a08f-0614ec8e320c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.314765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.315414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.315703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.315985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.317507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012f822-c32c-49a5-8013-11b2d94a19fe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.324022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.325689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:37.340014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:37.341006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:13:37.341935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:37.345072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 320/655] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:37.345850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.360830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.361794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.362290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.362779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.375326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.377281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:37.394384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:37.395407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:37.396272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:37.396272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:37.396272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:37.396272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:37.402834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 336/656] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:37.404063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.413836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.420792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.468678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c7b2f-e690-4188-8c36-935ad3949ee9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.507115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.507776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.508090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.508336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.509060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:37.509370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.510853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.517392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a43edf-1b7d-4d2d-a497-e4c99bac2e87 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.528594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 321/657] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:37.529638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.555822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.556446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.556757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.557030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.557806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:37.558113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.559523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.581399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 337/658] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:37.582646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.639822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bbf4f-fd98-46f6-a227-e20e4c82b676 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.649039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.662818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.679386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd2d5f-9b0e-4162-a619-86db3b3acf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.680012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd2d5f-9b0e-4162-a619-86db3b3acf04 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.680303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd2d5f-9b0e-4162-a619-86db3b3acf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.680600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd2d5f-9b0e-4162-a619-86db3b3acf04 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.681294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd2d5f-9b0e-4162-a619-86db3b3acf04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:37.681599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd2d5f-9b0e-4162-a619-86db3b3acf04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.682952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd2d5f-9b0e-4162-a619-86db3b3acf04 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.690699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180133eb-7333-438d-8568-60e78b6d6944 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.694517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 322/659] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:37.695761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979adaf-30f7-49d1-a4ed-9477f2edb503 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.696054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979adaf-30f7-49d1-a4ed-9477f2edb503 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.696335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979adaf-30f7-49d1-a4ed-9477f2edb503 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.701605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 323/660] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:37.702274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979adaf-30f7-49d1-a4ed-9477f2edb503 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.737954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.739763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.740049 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.740332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.747103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.748561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:37.760310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:37.761382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 338/661] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:37.762261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:37.826437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979adaf-30f7-49d1-a4ed-9477f2edb503 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.864172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.864848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.865131 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.865408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.869754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd996ca-8afc-4d80-9aee-78643f162731 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:37.894905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.907001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7492c-922b-4df8-abc6-5a7c23e80f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:37.907900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7492c-922b-4df8-abc6-5a7c23e80f87 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.908185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7492c-922b-4df8-abc6-5a7c23e80f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.908462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7492c-922b-4df8-abc6-5a7c23e80f87 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.916047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7492c-922b-4df8-abc6-5a7c23e80f87 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.917527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7492c-922b-4df8-abc6-5a7c23e80f87 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:37.921112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d5e7492c-922b-4df8-abc6-5a7c23e80f87 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:37.930570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:37.931596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:37.932630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 339/662] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:37.933239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05115298-8bd6-48c4-b2aa-6213f8da6c2d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:37.933239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05115298-8bd6-48c4-b2aa-6213f8da6c2d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:37.933719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05115298-8bd6-48c4-b2aa-6213f8da6c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:37.950826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:37.951145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-05115298-8bd6-48c4-b2aa-6213f8da6c2d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:37.952317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-05115298-8bd6-48c4-b2aa-6213f8da6c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:37.953051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.028711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 340/663] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:38.028711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05115298-8bd6-48c4-b2aa-6213f8da6c2d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.073650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 324/664] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:38.074728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.142027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05115298-8bd6-48c4-b2aa-6213f8da6c2d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.157275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.174182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.184659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.185295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.185582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.185863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.186618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:38.186929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:38.188362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.199888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6372f9-d5c2-4b6d-a1ad-1dd4ebf92ccf None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.201373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 341/665] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:38.202171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.242621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.243600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.244087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.244602 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.246023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:38.246535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:38.248500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.270859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 325/666] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:38.272052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.309269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ff482-6c9c-4f44-9bc4-be6142116368 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.345674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.346381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.346685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.347031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.358230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:38.364693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:38.365664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:38.366514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 342/667] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:38.367211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.388853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c1fdd-fc81-4133-8ae7-78318e192516 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.405656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.428485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.429369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.429800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.430240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.437111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.442251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:38.443782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:38.460032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:38.461168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:38.462272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:38.462272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:38.462272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:38.462272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:38.463801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 326/668] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:38.464778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.483694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f92f5-a869-4795-9418-c2c8356ed2e8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.520529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.521182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.521472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.521753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.522542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bc4b8f7545934151aafa986da871b944) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:38.522860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:38.524295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.530667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 343/669] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/role_assignments?user.id=bc4b8f7545934151aafa986da871b944&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:38.531451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.595162 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40727c-7a21-466e-b9bf-5537d6d773f9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.641932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.643034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.643034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.643507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.676390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.683587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ba91-b892-4860-bd54-f2f2748a801a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.721536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.735037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.735897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:38.736034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.736249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.736477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.737226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:38.737390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.737550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:38.740805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.748368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 344/670] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:38.749490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.861479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 327/671] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:38.862334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.873561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a85048-a94e-4418-a742-a04e3750d2c4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:38.907692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccb5d0-4942-460e-ba14-4b6b0769fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:38.908470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb5d0-4942-460e-ba14-4b6b0769fb71 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.908846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb5d0-4942-460e-ba14-4b6b0769fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.909222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb5d0-4942-460e-ba14-4b6b0769fb71 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.910052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccb5d0-4942-460e-ba14-4b6b0769fb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:38.910593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccb5d0-4942-460e-ba14-4b6b0769fb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:38.912189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccb5d0-4942-460e-ba14-4b6b0769fb71 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:38.928572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 345/672] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:38.928572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c812c-c3fd-44f1-bdef-28328f669ec3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:38.928572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c812c-c3fd-44f1-bdef-28328f669ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:38.928572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c812c-c3fd-44f1-bdef-28328f669ec3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:38.928572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 346/673] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:38.928991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c812c-c3fd-44f1-bdef-28328f669ec3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:38.952801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.970790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:38.982334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8b0ad-d53c-45ba-b5b7-5410e291e0a1 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.019992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603ec29-3db4-4ec9-a4b1-3d5ce9162a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.020684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603ec29-3db4-4ec9-a4b1-3d5ce9162a9e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.020964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603ec29-3db4-4ec9-a4b1-3d5ce9162a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.021239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603ec29-3db4-4ec9-a4b1-3d5ce9162a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.033337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603ec29-3db4-4ec9-a4b1-3d5ce9162a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.034835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603ec29-3db4-4ec9-a4b1-3d5ce9162a9e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.044495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c812c-c3fd-44f1-bdef-28328f669ec3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.085138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.085814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.086099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.086379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.099027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.100519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:39.119883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:39.121206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:39.122133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:39.122133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:39.122133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:39.122133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:39.122133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 347/674] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:39.122404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-3603ec29-3db4-4ec9-a4b1-3d5ce9162a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:39.172318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:39.173323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:39.174215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:39.174215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:39.174215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:39.174215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:39.174215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 328/675] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:39.175098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012503f8-9794-41f2-ac4e-affff765c412 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.175098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012503f8-9794-41f2-ac4e-affff765c412 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.175351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012503f8-9794-41f2-ac4e-affff765c412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.195974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-012503f8-9794-41f2-ac4e-affff765c412 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:39.197214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-012503f8-9794-41f2-ac4e-affff765c412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:39.206437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:39.239757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd04e2-01d0-4910-95ce-a52d66521326 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.270538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 329/676] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:39.271777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012503f8-9794-41f2-ac4e-affff765c412 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.280895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.281826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.282308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.282801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.290922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.292767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:39.305756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:39.306660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:39.307523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:39.314176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 348/677] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:39.314176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.351863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:39.384893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012503f8-9794-41f2-ac4e-affff765c412 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.422626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.423483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.423929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.424348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.425237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:39.425648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8832f4-626a-4fa7-8ab7-1fa415d94440 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.426336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.427914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.440404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 330/678] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:39.441255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.461282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:39.470393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.471013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.471295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.471670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.472392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:39.473236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.474799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.488144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 349/679] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:39.489988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.552356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cab735-8d21-44e5-82a0-1300e142211e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.585382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:39.593150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.594062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.594352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.594737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.604561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5c86b-e4c7-4b52-ae59-c95d1a17a69c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.605578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:39.610358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:39.611287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:39.612128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 331/680] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:39.612849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.647564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.648198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.648479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.648790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.649524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:39.649828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.651490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.664665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 350/681] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:39.665559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.725359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:39.736598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245d3a2-9672-434f-8a52-2c85ef69bc87 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.774177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.774852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.775143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.775419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.782684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0af170-38b5-48ea-85bb-f60bd4c05469 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.783501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.785544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:39.797241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:39.798133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:39.799098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 332/682] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:39.799600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.830752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.831408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.831689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.832886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.840244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.841732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:39.853288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:39.854275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:39.855199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:39.856632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:39.859220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 351/683] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:39.859972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.909349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4f7f6-6261-41bf-9c90-95189ba11bb2 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.944015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:39.944668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:39.944951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:39.945227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:39.945929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:39.946231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:39.947578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:39.961277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 333/684] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:39.962397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:39.966376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911ce3-3d70-4df5-be66-4324b8207225 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:39.981449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.005105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.005845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.006131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.006409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.007187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=ec1feaf015a847ffa719fd3a53de02e5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:40.007498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.008957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.015007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 352/685] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/role_assignments?user.id=ec1feaf015a847ffa719fd3a53de02e5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:40.015620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.084772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e69078-bf83-46b4-97d9-e441c8620c21 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:40.101670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.126440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.127481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.127975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.128458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.129795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:40.130344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.130611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193a40-e140-4294-90ea-bb7bc059cd35 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:40.132335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.147854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 334/686] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:40.149137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.170612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.171255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.171546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.171826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.172787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:40.172971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.174473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.196940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 353/687] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:40.198106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.237624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.284712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57afe6-a310-43f4-bfe6-ff2728384d63 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:40.323336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb672a1-5928-42a1-a28c-bdc6645ccaa1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:40.329540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.330373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.331058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.331483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.346355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.348680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.364508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.365298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.365678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.366070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.366948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=edf0a00538cf4e10b3bf266f48ea8cb9) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:40.367356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.368961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:40.369276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:40.370182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:40.371074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:40.371074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:40.371074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:40.371074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:40.373733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 335/688] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:40.374973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.385576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 354/689] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/role_assignments?user.id=edf0a00538cf4e10b3bf266f48ea8cb9&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:40.386297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.412941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.500809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a376c1b-57b0-42e0-8180-9950f5389e67 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:40.541569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151af78a-1084-46c6-965c-0caceb386aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.542882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151af78a-1084-46c6-965c-0caceb386aae None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.543333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151af78a-1084-46c6-965c-0caceb386aae None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.543756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151af78a-1084-46c6-965c-0caceb386aae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.634848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.640470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7b304-7e28-4083-8b51-08a2fcfd1e51 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:40.644295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151af78a-1084-46c6-965c-0caceb386aae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.647146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151af78a-1084-46c6-965c-0caceb386aae None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.683869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.693218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cd465-9261-49d5-8912-7f2c21a7ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.694339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cd465-9261-49d5-8912-7f2c21a7ba0a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.694773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cd465-9261-49d5-8912-7f2c21a7ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.696138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cd465-9261-49d5-8912-7f2c21a7ba0a None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.697035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cd465-9261-49d5-8912-7f2c21a7ba0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:40.697496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cd465-9261-49d5-8912-7f2c21a7ba0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.699011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cd465-9261-49d5-8912-7f2c21a7ba0a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.716472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 355/690] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:40.723473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8fae5-f092-4a6b-a6e4-5b2bd1d1521c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.723771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8fae5-f092-4a6b-a6e4-5b2bd1d1521c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.724056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8fae5-f092-4a6b-a6e4-5b2bd1d1521c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.729298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 356/691] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:40.729298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8fae5-f092-4a6b-a6e4-5b2bd1d1521c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.787716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 336/692] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:40.789257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa1a6-e1c1-4f8e-b6a6-967af268847a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.789685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa1a6-e1c1-4f8e-b6a6-967af268847a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.790101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa1a6-e1c1-4f8e-b6a6-967af268847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.808024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ba8fa1a6-e1c1-4f8e-b6a6-967af268847a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:40.809239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ba8fa1a6-e1c1-4f8e-b6a6-967af268847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:40.859764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8fae5-f092-4a6b-a6e4-5b2bd1d1521c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:40.884281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 337/693] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:40.885386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fa1a6-e1c1-4f8e-b6a6-967af268847a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.913283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:40.913937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:40.914219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:40.914496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:40.929591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:40.931036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:40.944414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:40.945714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:40.946872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:40.950284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 357/694] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:40.950911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.952668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:40.981506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:40.999869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fa1a6-e1c1-4f8e-b6a6-967af268847a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.037155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.038002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.038204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.038487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.049235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.051319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:41.068340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:41.069445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:41.070462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:41.070462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:41.070462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:41.070462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:41.073486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ff325-ce2e-4b55-bb62-1dbc2271881d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.074141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 338/695] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:41.074981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.120600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.121263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.122091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.122385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.136172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.137665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:41.152791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:41.153884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:41.155004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:41.155004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:41.155004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:41.155004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:41.155004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 358/696] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:41.155004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.197487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc92f0-ba44-4b88-a4eb-4495b3df3ce1 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.207691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:41.227920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:41.241426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.242479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.243077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.243554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.244637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:41.245266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.247010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.263077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bc98d-f31a-4356-8f34-8e907d7eabd1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.268395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 339/697] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:41.270114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.304913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.305455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.305802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.306088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.306844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:41.307152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.308619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.321856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 359/698] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:41.322896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.389974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8460bf-af7c-4af7-b004-1f5eebd90b18 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.427859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9e9a2-997b-4b88-940f-27f2c883d5ef None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.429572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.430518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.430996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.431465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.432363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:41.432884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.434431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.453021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 340/699] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:41.454178 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.459053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:41.465401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.466027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.466316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.466609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.490874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:41.538930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.540577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.573193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd75507-4b33-435a-8635-13b475054991 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.610906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.612182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.612763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.613258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.620920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.622718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:41.636687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:41.637574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:41.638480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:41.641076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 341/700] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:41.642164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.651120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 360/701] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:41.652034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.683716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:41.748459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:41.756870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbb3f3-0549-428b-9bea-2bac32ff2ff3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.768661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b6013-a521-4f5e-a726-c76096fc92dc None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:41.792609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.793075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.793364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.793643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.794398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:41.794711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.796141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.809424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 361/702] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:41.810201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:41.810343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.811183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:41.811661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:41.812129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:41.813175 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:41.814939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:41.848816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 342/703] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:13:41 2022] POST /identity/v3/users => generated 254 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:41.848816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:41.973146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:41.990611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6779b91-8dd1-4b6d-a145-f37f385978d7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.048679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc2ea7-2030-47f2-a6ba-cb70fcf9a233 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.052953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.053616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.053906 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.056598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.075639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.077140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:42.093193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:42.094185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:42.095107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:42.095107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:42.095107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:42.095107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:42.095107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 362/704] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:42.100763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:42.107794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b8b80-3dc9-4f02-8c5b-57471917a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.108422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b8b80-3dc9-4f02-8c5b-57471917a017 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.108735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b8b80-3dc9-4f02-8c5b-57471917a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.109010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b8b80-3dc9-4f02-8c5b-57471917a017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.110325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b8b80-3dc9-4f02-8c5b-57471917a017 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=e35b790a4a5b4306a5537984d0b449e6) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:42.110642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b8b80-3dc9-4f02-8c5b-57471917a017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.113019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b8b80-3dc9-4f02-8c5b-57471917a017 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.122390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 343/705] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:41 2022] GET /identity/v3/role_assignments?user.id=e35b790a4a5b4306a5537984d0b449e6&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:42.176579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f30a9-b908-4819-a99e-f5abb00d6204 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.176904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f30a9-b908-4819-a99e-f5abb00d6204 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.177321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f30a9-b908-4819-a99e-f5abb00d6204 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.178944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 344/706] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:42.184937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fca634-924e-43f3-9f5f-dfccfe5309fd None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.185232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fca634-924e-43f3-9f5f-dfccfe5309fd None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.185488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fca634-924e-43f3-9f5f-dfccfe5309fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.205330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:42.218915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-99fca634-924e-43f3-9f5f-dfccfe5309fd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:42.220770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-99fca634-924e-43f3-9f5f-dfccfe5309fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:42.244083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33e9a8-0380-4fdc-ba06-257aedaca4d4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.288143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.289168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.289652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.290962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.291869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:42.292624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.294351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.311500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 363/707] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:42.314431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 345/708] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:42.316776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:42.320929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:42.340956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fca634-924e-43f3-9f5f-dfccfe5309fd None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:42.430694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7720f5-403f-45bf-8cf8-daf4c02460e9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.468786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.469444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.469727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.470004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.471220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fca634-924e-43f3-9f5f-dfccfe5309fd None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.478382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.479896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:42.493028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:42.493948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:42.495453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 364/709] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:42.500451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:42.512144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.512832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.513111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.513381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.514376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.515742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.522297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:42.532739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:42.534266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:42.536532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 346/710] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jul 29 06:13:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:42.541033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:42.587180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:42.621873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454914f-0895-47d7-ba81-c6dcac288837 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.651002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e84302-1470-43da-92e2-69deac1fc9bb None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.659409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.660192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.660482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.660795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.661898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:42.662016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.663597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.680580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 365/711] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:42.683747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:42.714196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.714832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.715111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.715389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.716172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:42.716481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.717963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.736887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 347/712] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 29 06:13:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:42.742254 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:42.797238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacd2f3-1e97-4409-baab-e014e39546e3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.822355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:42.836145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa38b5-7af6-413f-89d5-05862089777d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.836853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa38b5-7af6-413f-89d5-05862089777d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.837142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa38b5-7af6-413f-89d5-05862089777d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.837420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa38b5-7af6-413f-89d5-05862089777d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.838270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa38b5-7af6-413f-89d5-05862089777d None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bbe64416672b408f8cdf93220cc31ef5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:42.838587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa38b5-7af6-413f-89d5-05862089777d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.840076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa38b5-7af6-413f-89d5-05862089777d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.846465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 366/713] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:42 2022] GET /identity/v3/role_assignments?user.id=bbe64416672b408f8cdf93220cc31ef5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:42.863950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:42.896912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f5ce-36ab-4ce4-8eae-aba6201e8381 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.897314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f5ce-36ab-4ce4-8eae-aba6201e8381 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.897616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f5ce-36ab-4ce4-8eae-aba6201e8381 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.899431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 367/714] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:42.904760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973dac-7726-4757-b7f7-49806fbfe65f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.905053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973dac-7726-4757-b7f7-49806fbfe65f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.905455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973dac-7726-4757-b7f7-49806fbfe65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.908304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7bd1-bb46-4083-b996-e12dc1c32a5b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:42.939917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-61973dac-7726-4757-b7f7-49806fbfe65f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:42.945343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-61973dac-7726-4757-b7f7-49806fbfe65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:42.976839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:42.977789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:42.978372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:42.979860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:42.980801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:42.982547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:42.984620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:42.998937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 348/715] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 29 06:13:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:43.057132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 368/716] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:43.083609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:43.217383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77998ca-737d-4d7d-abf6-1270855f0ce6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:43.248694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:43.259276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:43.259926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:43.260206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:43.260478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] PATH_INFO: `/v3/services/network` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:43.261145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:43.262503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:43.271480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:43.272467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:43.273354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 349/717] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:43 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:43.278890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973dac-7726-4757-b7f7-49806fbfe65f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:43.400530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973dac-7726-4757-b7f7-49806fbfe65f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:43.420599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:43.440926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1eb65-93c9-4de6-beb0-6b14d643cb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:43.442104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1eb65-93c9-4de6-beb0-6b14d643cb66 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:43.442386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1eb65-93c9-4de6-beb0-6b14d643cb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:43.442663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1eb65-93c9-4de6-beb0-6b14d643cb66 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:43.443341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1eb65-93c9-4de6-beb0-6b14d643cb66 None admin] RBAC: Adding query filter params (name=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:43.443648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1eb65-93c9-4de6-beb0-6b14d643cb66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:43.445038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1eb65-93c9-4de6-beb0-6b14d643cb66 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:43.449966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 369/718] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:13:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:43.457702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:43.588378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fa0a4-9d2d-46e4-a91f-825c8901544e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:43.629417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea7e4d-3974-41b6-9679-09b0a83e9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:43.630046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea7e4d-3974-41b6-9679-09b0a83e9347 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:43.630326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea7e4d-3974-41b6-9679-09b0a83e9347 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:43.630600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea7e4d-3974-41b6-9679-09b0a83e9347 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:43.631272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea7e4d-3974-41b6-9679-09b0a83e9347 None admin] RBAC: Adding query filter params (type=network) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:43.631575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea7e4d-3974-41b6-9679-09b0a83e9347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:43.633029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea7e4d-3974-41b6-9679-09b0a83e9347 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:43.637892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 350/719] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:13:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:44.436712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2a892-3e49-413a-b0cc-b6718c8f43b8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:44.437568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2a892-3e49-413a-b0cc-b6718c8f43b8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:44.437996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2a892-3e49-413a-b0cc-b6718c8f43b8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:44.439740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 370/720] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:44.445897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5048fe2-c10b-4fe3-87a2-b5d1dd530ea2 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:44.446652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5048fe2-c10b-4fe3-87a2-b5d1dd530ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:44.447101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5048fe2-c10b-4fe3-87a2-b5d1dd530ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:44.466419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d5048fe2-c10b-4fe3-87a2-b5d1dd530ea2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:44.467828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d5048fe2-c10b-4fe3-87a2-b5d1dd530ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:44.521342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:44.539713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 351/721] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 06:13:44.562809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2a892-3e49-413a-b0cc-b6718c8f43b8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:44.696816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2a892-3e49-413a-b0cc-b6718c8f43b8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:44.885391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:44.886122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:44.886411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:44.886689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:44.899377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:44.904740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:44.905645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:44.906464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 371/722] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:44.912627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5048fe2-c10b-4fe3-87a2-b5d1dd530ea2 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:45.047027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5048fe2-c10b-4fe3-87a2-b5d1dd530ea2 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:45.087603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c93af-9acb-4dc5-9b81-a00a26c4ceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:45.088241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c93af-9acb-4dc5-9b81-a00a26c4ceb9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.088573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c93af-9acb-4dc5-9b81-a00a26c4ceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.088828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c93af-9acb-4dc5-9b81-a00a26c4ceb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.089550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c93af-9acb-4dc5-9b81-a00a26c4ceb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:45.089858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c93af-9acb-4dc5-9b81-a00a26c4ceb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:45.091232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c93af-9acb-4dc5-9b81-a00a26c4ceb9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:45.099898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 352/723] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:45.105342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:45.162652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205bfb6-2020-435d-90cc-b451c106770b None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.162966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205bfb6-2020-435d-90cc-b451c106770b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.163507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205bfb6-2020-435d-90cc-b451c106770b None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.166838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 353/724] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:45.174849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d645cc-7f79-4b4a-9246-c8a32ace87f8 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.175126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d645cc-7f79-4b4a-9246-c8a32ace87f8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.175444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d645cc-7f79-4b4a-9246-c8a32ace87f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.177492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:45.210966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b8d645cc-7f79-4b4a-9246-c8a32ace87f8 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:45.212223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b8d645cc-7f79-4b4a-9246-c8a32ace87f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:45.263387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625736-a0ed-4167-8cde-90846d254821 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:45.304859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:45.305445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.305726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.306002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.326671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:45.328818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:45.337085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:45.357393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:45.358447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:45.359342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:45.359342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:45.359342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:45.359342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:45.359342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 372/725] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:45.370210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:45.380893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 354/726] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:45.424416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d645cc-7f79-4b4a-9246-c8a32ace87f8 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:45.521810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:45.532077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0625cec-0c0a-4b62-8b34-a4bbd79b7dd9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:45.572066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f0502-96bc-4801-ac9e-6e794e0ff033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:45.572958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f0502-96bc-4801-ac9e-6e794e0ff033 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.573395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f0502-96bc-4801-ac9e-6e794e0ff033 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.573944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f0502-96bc-4801-ac9e-6e794e0ff033 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.574845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f0502-96bc-4801-ac9e-6e794e0ff033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:45.575364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f0502-96bc-4801-ac9e-6e794e0ff033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:45.577024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f0502-96bc-4801-ac9e-6e794e0ff033 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:45.589057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d645cc-7f79-4b4a-9246-c8a32ace87f8 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:45.592173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 373/727] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:45.593763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162047-3224-4826-bbb4-ec1f585e12cf None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.594247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162047-3224-4826-bbb4-ec1f585e12cf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.594711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162047-3224-4826-bbb4-ec1f585e12cf None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.600804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 374/728] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:45.601973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc162047-3224-4826-bbb4-ec1f585e12cf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:45.630008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc552bfc-4461-4bd3-b985-bac3a55a5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:45.630801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552bfc-4461-4bd3-b985-bac3a55a5f17 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.631226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552bfc-4461-4bd3-b985-bac3a55a5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.631580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552bfc-4461-4bd3-b985-bac3a55a5f17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.652444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc552bfc-4461-4bd3-b985-bac3a55a5f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-fc552bfc-4461-4bd3-b985-bac3a55a5f17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:45.660891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:45.662065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:45.663044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:45.672428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 355/729] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:45.674105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466fa4a-dd71-46bf-acd2-0ee4d5511a63 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.674710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466fa4a-dd71-46bf-acd2-0ee4d5511a63 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.675176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466fa4a-dd71-46bf-acd2-0ee4d5511a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.707735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8466fa4a-dd71-46bf-acd2-0ee4d5511a63 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:45.710402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8466fa4a-dd71-46bf-acd2-0ee4d5511a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:45.721269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc162047-3224-4826-bbb4-ec1f585e12cf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:45.762944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:45.763653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.763938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.764212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.766518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:45.773521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:45.774960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:45.787625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:45.788694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 375/730] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:13:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:45.789635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:45.809120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:45.822193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 356/731] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:45.823723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80554b3f-c6fe-40c6-8d2b-2d71f9edd2e5 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.824172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80554b3f-c6fe-40c6-8d2b-2d71f9edd2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.824461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80554b3f-c6fe-40c6-8d2b-2d71f9edd2e5 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.826382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 357/732] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:45.827457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80554b3f-c6fe-40c6-8d2b-2d71f9edd2e5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:45.918171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416481dc-6dfd-46d4-bcb6-9379527388a5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:45.952253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80554b3f-c6fe-40c6-8d2b-2d71f9edd2e5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:45.966703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849c3b3-39e9-46a4-9dba-fdbe1f4aa6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:45.967337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c3b3-39e9-46a4-9dba-fdbe1f4aa6da None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.967627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c3b3-39e9-46a4-9dba-fdbe1f4aa6da None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.967900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c3b3-39e9-46a4-9dba-fdbe1f4aa6da None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:45.968635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849c3b3-39e9-46a4-9dba-fdbe1f4aa6da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:45.968939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849c3b3-39e9-46a4-9dba-fdbe1f4aa6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:45.970336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849c3b3-39e9-46a4-9dba-fdbe1f4aa6da None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:45.980123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 376/733] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:45.981732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57edd6-7f4c-4f04-afdb-f6a82f5f49e0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:45.981732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57edd6-7f4c-4f04-afdb-f6a82f5f49e0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:45.982546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57edd6-7f4c-4f04-afdb-f6a82f5f49e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.000239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.000613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0d57edd6-7f4c-4f04-afdb-f6a82f5f49e0 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:46.000902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.001182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.001457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.001778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0d57edd6-7f4c-4f04-afdb-f6a82f5f49e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:46.002245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:46.002546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.004674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.023871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 358/734] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 29 06:13:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:46.029105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.055920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.075726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 377/735] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:46.076812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57edd6-7f4c-4f04-afdb-f6a82f5f49e0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.085909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.147816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05579cb-0dc4-4d06-b9c0-d4604a4b51a8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.186754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.187514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.187802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.188205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.190971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57edd6-7f4c-4f04-afdb-f6a82f5f49e0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.200476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:46.205322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:46.206507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:46.207378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 359/736] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:46.208146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.236930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.237583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.238404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.238690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.249993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.251448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:46.267263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:46.268254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:46.269330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:46.269330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:46.269330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:46.269330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:46.269330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 378/737] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:46.269803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.305852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.322800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c862601-5482-4ecc-bb7c-520390271356 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.335404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.357435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.358061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.358343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.358618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.380462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffb805-e346-4c38-9fa0-7bfac7e16a1e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.412999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.413507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.413791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.414084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.414786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.416188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.428802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.430223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.432576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 379/738] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:13:46 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:46.433463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.531574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.553399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 360/739] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 06:13:46.554330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.564584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b1ac1-e87e-481c-af02-794266b9e90b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.579315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.614807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46efca7-6059-4ee5-8f77-522476a8ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.615355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46efca7-6059-4ee5-8f77-522476a8ed47 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.615704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46efca7-6059-4ee5-8f77-522476a8ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.616057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46efca7-6059-4ee5-8f77-522476a8ed47 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.616805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46efca7-6059-4ee5-8f77-522476a8ed47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:46.617109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46efca7-6059-4ee5-8f77-522476a8ed47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.618472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46efca7-6059-4ee5-8f77-522476a8ed47 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.633331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 380/740] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:46.634501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bbd4a-45cc-4401-bbe6-16d3c303cbdf None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.634787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bbd4a-45cc-4401-bbe6-16d3c303cbdf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.635063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bbd4a-45cc-4401-bbe6-16d3c303cbdf None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.638248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 381/741] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:46.639015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bbd4a-45cc-4401-bbe6-16d3c303cbdf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.670128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e17e7-a611-46ca-ba92-29b674459d19 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.708861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.709489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.709772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.710047 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.710730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:46.711037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.712496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.733593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 361/742] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:46.733593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.751549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bbd4a-45cc-4401-bbe6-16d3c303cbdf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.783861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.787514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9dae3-d48c-41ef-b802-aaa3c74653fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.788170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9dae3-d48c-41ef-b802-aaa3c74653fb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.788454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9dae3-d48c-41ef-b802-aaa3c74653fb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.788765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9dae3-d48c-41ef-b802-aaa3c74653fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.798975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9dae3-d48c-41ef-b802-aaa3c74653fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.800595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9dae3-d48c-41ef-b802-aaa3c74653fb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b8f9dae3-d48c-41ef-b802-aaa3c74653fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:46.815214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:46.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:46.817905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:46.817905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:46.817905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:46.817905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:46.817905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 382/743] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:46.818882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f6c79-32c7-4373-a062-4ad7858354a1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.819175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f6c79-32c7-4373-a062-4ad7858354a1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.819463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f6c79-32c7-4373-a062-4ad7858354a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.837877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-533f6c79-32c7-4373-a062-4ad7858354a1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:46.839075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-533f6c79-32c7-4373-a062-4ad7858354a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:46.848815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3142c-ac4d-4e5d-9958-35de786658f2 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:46.859116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:46.888406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:46.889011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:46.889406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:46.889694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:46.902416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:46.903876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:46.911517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 383/744] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 06:13:46.912672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f6c79-32c7-4373-a062-4ad7858354a1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:46.930404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:46.931341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:46.932211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:46.932211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:46.932211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:46.932211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:46.938843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 362/745] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:46.939893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.029402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f6c79-32c7-4373-a062-4ad7858354a1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.046512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:47.051934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e5eed-70f7-4242-bf1b-9a6589819211 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.068395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.069098 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.069377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.069650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.076893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.078318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.087680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.088407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.088792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.089148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.089919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:47.090306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:47.091177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:47.092069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:47.093132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:47.093664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.097031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 384/746] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:47.097886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.106797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 363/747] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:47.107641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.119759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:47.209075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a3299-5371-4842-a6cb-80bde421e66c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.216776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f571d-3019-4cf1-8edc-022726742c8b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.246254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.246882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.247170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.247451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.248141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.249537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.254749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.255374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.255695 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.255975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.256735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:47.257084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.258714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.272610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 385/748] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:13:47 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:47.272610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.286481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 364/749] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:47.287486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.308971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:47.377896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:47.412501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287041-4827-42bb-a274-1cfa09e5b298 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.421443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60ba3f-abd4-4161-b2ea-d35fc49e36f9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.451472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.452109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.452395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.452731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.453528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:47.453841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.455262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.463475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.464145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.464504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.464903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.474045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.475797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 386/750] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:47.475993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.476757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:47.493165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:47.494211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 365/751] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:47.495273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.590514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:47.616374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0f6a6-6921-4e08-9af3-8cf95175712e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.622949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d607ba0-2582-4da2-b0c6-7bd4f06174e6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.659406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:47.660273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.661322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.661783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.662223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.666840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.668391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.668704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.668981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.678361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.680570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:47.697862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:47.699033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:47.700097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:47.703719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 387/752] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:47.704730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.762862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.765274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.838952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2dcf7-b017-424a-9b8d-55819bd98f71 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:47.869216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:47.875284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:47.875963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:47.876247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:47.876524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:47.877390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:47.877683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:47.879234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:47.896566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 388/753] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:47.896566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.906112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 366/754] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:13:47.906969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:47.967712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:48.026521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669a258-bddd-407c-bbfa-b42fba0caa94 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.036761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e626f-ef34-4594-81e5-cfb4bfc4c722 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.065518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.067006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.067369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.067722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.068620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=a4141b562f1442efaa032122666724df) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:48.069170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.070770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.080871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 367/755] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/role_assignments?user.id=a4141b562f1442efaa032122666724df&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:48.080871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:48.093867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.094808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.095246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.095683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.096688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:48.097193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.098869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.135620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 389/756] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:13:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:48.141580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:48.171886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:48.304020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90da428-9485-4b12-bdac-4bdfc2c90094 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.317613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db412f7-6ffd-447e-b530-ba6dca44db3e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.330389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:48.356870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.357700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.358076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.358449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.381988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.384353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.386015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.386767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.388324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.388741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:48.413409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:48.414718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:48.415707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:48.415707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:48.415707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:48.415707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:48.416127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.417660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.419291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 368/757] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:48.429265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:48.451311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:48.453491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:48.455885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:48.455885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:48.455885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:48.455885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:48.455885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 390/758] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:48.458741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:48.567917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:48.576768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca45297-d64b-4433-baf1-25c9ccd7abbd None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.611001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c44dc4-2f09-40e0-9567-cacc838a6df7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.616470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd390346-82d0-4385-b4c2-acf27e69a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.617243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd390346-82d0-4385-b4c2-acf27e69a140 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.617663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd390346-82d0-4385-b4c2-acf27e69a140 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.618019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd390346-82d0-4385-b4c2-acf27e69a140 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.618915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd390346-82d0-4385-b4c2-acf27e69a140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:48.619329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd390346-82d0-4385-b4c2-acf27e69a140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.621010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd390346-82d0-4385-b4c2-acf27e69a140 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.636765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 369/759] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:48.638413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7eb0-b89f-4b94-b539-6a50fdba59e5 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.638704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7eb0-b89f-4b94-b539-6a50fdba59e5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.639134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7eb0-b89f-4b94-b539-6a50fdba59e5 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.640417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 370/760] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:48.647404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c7eb0-b89f-4b94-b539-6a50fdba59e5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:48.652765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748deaa-d86c-4633-9284-345afccd758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.653390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748deaa-d86c-4633-9284-345afccd758e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.653676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748deaa-d86c-4633-9284-345afccd758e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.653957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748deaa-d86c-4633-9284-345afccd758e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.654648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748deaa-d86c-4633-9284-345afccd758e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:48.654966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748deaa-d86c-4633-9284-345afccd758e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.656337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748deaa-d86c-4633-9284-345afccd758e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.669758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 391/761] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:48.671172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cb877-a5fd-40d2-afa9-a1c9e7323a83 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.671474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cb877-a5fd-40d2-afa9-a1c9e7323a83 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.671745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cb877-a5fd-40d2-afa9-a1c9e7323a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.689832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9f8cb877-a5fd-40d2-afa9-a1c9e7323a83 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:48.691233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9f8cb877-a5fd-40d2-afa9-a1c9e7323a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:48.710926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:48.786225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c7eb0-b89f-4b94-b539-6a50fdba59e5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.791930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 392/762] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:13:48.793106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cb877-a5fd-40d2-afa9-a1c9e7323a83 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:48.837765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.838672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.838964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.839661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.849222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.851223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:48.867173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:48.868101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:48.869034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:48.870153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 371/763] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:48.872350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:48.891487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:48.922401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cb877-a5fd-40d2-afa9-a1c9e7323a83 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:48.970152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:48.970897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:48.971245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:48.971583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:48.980971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:48.982666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:48.997837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:48.998951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:49.000075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:49.001297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 393/764] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:49.002408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.009114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcdf43-7d7c-4775-a179-97c5c4cce7c9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.017325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:49.064320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.065577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.066612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.067109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.068230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.070725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:49.085022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:49.086231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:49.087275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:49.095015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 372/765] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jul 29 06:13:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:49.095015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.144918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429ec82-f189-4ebe-8a36-1d62b9bc092c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.188158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:49.193231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.194524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.195112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.195545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.196643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:49.197108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.198977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.213915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6d5d0-6f29-4663-b7b6-a02a0c166e99 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.217358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 394/766] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:49.218368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.255580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.256407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.256806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.257292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.258252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:49.258654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.260260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.274384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 373/767] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:49.275219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.317579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:49.354968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff61e2-036f-4c59-85cd-d5e024c216be None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.390825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.391438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.391719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.391996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.392703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:49.393018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.394907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.400342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 395/768] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:49.400931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.410364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20307be1-4d7e-41c9-aae5-ae66d57337f2 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.458588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.460012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.460788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.461268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.462540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=e35b790a4a5b4306a5537984d0b449e6) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:49.463421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.465588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.475368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 374/769] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/role_assignments?user.id=e35b790a4a5b4306a5537984d0b449e6&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:49.476335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.479224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:49.528827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67138359-e11f-4248-9454-0a56c7d13f36 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.580279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636979b-a722-4c13-8c8d-d17b79be9eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.581426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636979b-a722-4c13-8c8d-d17b79be9eb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.582284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636979b-a722-4c13-8c8d-d17b79be9eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.582642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636979b-a722-4c13-8c8d-d17b79be9eb5 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/bbe64416672b408f8cdf93220cc31ef5/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.585577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636979b-a722-4c13-8c8d-d17b79be9eb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=bbe64416672b408f8cdf93220cc31ef5, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.587593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636979b-a722-4c13-8c8d-d17b79be9eb5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.591147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:49.601177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98be618-f71f-4c9a-b4c3-bfe0c157666d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.606989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 396/770] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:49 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/bbe64416672b408f8cdf93220cc31ef5/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:49.608370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b98189-d7dc-4fdd-b6c3-bad6f28c8ae3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.608773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b98189-d7dc-4fdd-b6c3-bad6f28c8ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.609062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b98189-d7dc-4fdd-b6c3-bad6f28c8ae3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.612768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 397/771] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:49.617925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e12767-f77a-46b6-bc78-421db22c6d2b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.618229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e12767-f77a-46b6-bc78-421db22c6d2b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.618514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e12767-f77a-46b6-bc78-421db22c6d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.646734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e7e12767-f77a-46b6-bc78-421db22c6d2b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:49.649308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e7e12767-f77a-46b6-bc78-421db22c6d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:49.654929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.655913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.656451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.656989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.657880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:49.659105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.660913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.669247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 375/772] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:49.674681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.748685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 398/773] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:13:49.770390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e12767-f77a-46b6-bc78-421db22c6d2b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.781075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:49.822636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90990a-c2de-4544-b862-ccbb25f535a9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.860910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.861950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.862444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.862984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.864186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] RBAC: Adding query filter params (interface=public, service_id=3d201c08309f4292aa6c2c99637ef6f1, region_id=RegionOne) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:49.864788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.866435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.875990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 376/774] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/endpoints?service_id=3d201c08309f4292aa6c2c99637ef6f1&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:49.882405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.926863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e12767-f77a-46b6-bc78-421db22c6d2b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:49.958822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:49.974794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:49.975173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:49.975461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:49.975744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] PATH_INFO: `/v3/services/network` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:49.976384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:49.977819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:49.990078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:49.991106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:49.992341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 399/775] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:49.997525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:49.999491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d9ced-d3cc-40e5-9000-9a04c62f45d1 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:50.031116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:50.038932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:50.039889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.040381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.040907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.042003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:50.043667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:50.049833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 377/776] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:50.131600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54235909-3cf7-4342-8396-0a72d71d43ed None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:50.186547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:50.187553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.188069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.188570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.189468 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] RBAC: Adding query filter params (name=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:50.189986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:50.191696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:50.199354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 400/777] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:13:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:50.205114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:50.338159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241688b3-7b20-41d8-8df0-d089140bf19c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:50.375976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf55fff-1a81-46a4-b85c-a4efe4f4f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:50.376999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55fff-1a81-46a4-b85c-a4efe4f4f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.377513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55fff-1a81-46a4-b85c-a4efe4f4f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.378037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55fff-1a81-46a4-b85c-a4efe4f4f6b7 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.379465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf55fff-1a81-46a4-b85c-a4efe4f4f6b7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:50.380005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf55fff-1a81-46a4-b85c-a4efe4f4f6b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:50.381716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf55fff-1a81-46a4-b85c-a4efe4f4f6b7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:50.387374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 378/778] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:13:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:50.392806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:50.451002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68521c-720a-4a61-9673-09772f4be69a None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.451454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68521c-720a-4a61-9673-09772f4be69a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.451850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68521c-720a-4a61-9673-09772f4be69a None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.453423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 379/779] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:50.455193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:50.462973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b471a1-3dc5-4fa6-bae7-1a353aa26960 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.463387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b471a1-3dc5-4fa6-bae7-1a353aa26960 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.463672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b471a1-3dc5-4fa6-bae7-1a353aa26960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.484912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d1b471a1-3dc5-4fa6-bae7-1a353aa26960 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:50.486520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d1b471a1-3dc5-4fa6-bae7-1a353aa26960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:50.540498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca765f5a-3cfb-452b-880d-9384648f2426 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:50.570772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:50.598691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 380/780] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:13:50.608489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:50.609339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.609797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.610244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.611159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] RBAC: Adding query filter params (interface=public, service_id=db65f6b0e31641f3a980054f067b14e6, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:50.611624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:50.613257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:50.621017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b471a1-3dc5-4fa6-bae7-1a353aa26960 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:50.627846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 401/781] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:13:50 2022] GET /identity/v3/endpoints?service_id=db65f6b0e31641f3a980054f067b14e6&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:50.634034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:50.749203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76bf12-c8dc-445d-b6eb-0fd0f23adc33 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:50.788929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:50.790418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.790859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.791285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.792158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:50.794191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:50.800772 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:50.801062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 402/782] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:13:50 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:50.908584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b471a1-3dc5-4fa6-bae7-1a353aa26960 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:50.962269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:50.962937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:50.963223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:50.963503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:50.992838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:50.997744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:50.998643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:50.999467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 381/783] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:51.004856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:51.111747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86cf4d-5fd3-473b-9da8-0e2e54347f87 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:51.152015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:51.152689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:51.152978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:51.153258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:51.153993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:51.154361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:51.155809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:51.165144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 403/784] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:51.177706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:51.237667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:51.294022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3518ec-1845-40ed-a923-3fa6a1914a55 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:51.340389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:51.341070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:51.341351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:51.341626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:51.355021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:51.356564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:51.381097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:51.382044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:51.382928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:51.382928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:51.382928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:51.382928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:51.382928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 382/785] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:51.387774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:51.447435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:51.526784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3456f-13a4-4c8b-a0dd-da686bc81ab4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:51.572288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:51.572945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:51.573233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:51.573509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:51.574197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:51.574564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:51.575928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:51.591808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 404/786] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:51.597020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:51.741776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:51.745497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3135a-1f49-4297-80f5-994fdb5991ab None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:51.799042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065e57d-5382-4a76-b06d-3393ef6b444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:51.799686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065e57d-5382-4a76-b06d-3393ef6b444e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:51.799968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065e57d-5382-4a76-b06d-3393ef6b444e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:51.800936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065e57d-5382-4a76-b06d-3393ef6b444e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:51.817718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065e57d-5382-4a76-b06d-3393ef6b444e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:51.819173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065e57d-5382-4a76-b06d-3393ef6b444e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-7065e57d-5382-4a76-b06d-3393ef6b444e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:51.834421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:51.835414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:51.836365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 383/787] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:13:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:51.841241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:51.876010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac21e2-68b0-4354-b8b0-5586c5a5eb72 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:51.876357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac21e2-68b0-4354-b8b0-5586c5a5eb72 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:51.876604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac21e2-68b0-4354-b8b0-5586c5a5eb72 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:51.877937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 384/788] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:51.884782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82765-09b9-47c3-aaef-13c6e7c1867c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:51.885070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82765-09b9-47c3-aaef-13c6e7c1867c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:51.885352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82765-09b9-47c3-aaef-13c6e7c1867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:51.920651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d4d82765-09b9-47c3-aaef-13c6e7c1867c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:51.921853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d4d82765-09b9-47c3-aaef-13c6e7c1867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:51.998413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d5cc9-d72c-4271-96cc-ec0719b1e08b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:51.999901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:52.054125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 385/789] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:13:52.063390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.064023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.064311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.064618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.065414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:52.066259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:52.067690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:52.098182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82765-09b9-47c3-aaef-13c6e7c1867c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.099087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 405/790] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 29 06:13:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:52.104083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.270277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:52.282991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52646dc-4f8b-4e1a-93e5-8d8a4ac630d7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:52.299140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82765-09b9-47c3-aaef-13c6e7c1867c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:52.328372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92080df7-fab3-4116-93f0-34ba4a8adb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.329067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080df7-fab3-4116-93f0-34ba4a8adb62 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.329355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080df7-fab3-4116-93f0-34ba4a8adb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.329734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080df7-fab3-4116-93f0-34ba4a8adb62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.345066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.345816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.346105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.346384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.347535 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92080df7-fab3-4116-93f0-34ba4a8adb62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:52.357923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:52.358118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92080df7-fab3-4116-93f0-34ba4a8adb62 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:52.363975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:52.365079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:52.365908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 386/791] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:52.382213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-92080df7-fab3-4116-93f0-34ba4a8adb62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:52.383022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:52.385035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 406/792] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647e20c-5227-4ee2-a411-bf076ef98188 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647e20c-5227-4ee2-a411-bf076ef98188 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.387111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647e20c-5227-4ee2-a411-bf076ef98188 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.391531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 407/793] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:52.393131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f0695-b9f7-412c-820a-8e8c2c21659e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.393424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f0695-b9f7-412c-820a-8e8c2c21659e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.393711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f0695-b9f7-412c-820a-8e8c2c21659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.413040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-654f0695-b9f7-412c-820a-8e8c2c21659e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:52.414362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-654f0695-b9f7-412c-820a-8e8c2c21659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:52.423250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:52.496652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 408/794] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:13:52.497675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f0695-b9f7-412c-820a-8e8c2c21659e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.498109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f16e0-f485-4206-8b12-1989a84b23a8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:52.541028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.542080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.542439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.543201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.544272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:52.544695 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:52.547035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:52.563939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 387/795] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:52.564946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.585318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:52.630310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f0695-b9f7-412c-820a-8e8c2c21659e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:52.674106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.674978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.676029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.676449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.677298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:52.677812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:52.679352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:52.696462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 409/796] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:52.697621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.700726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:52.707416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c243d5-03bf-4db5-9e98-e1dda59094ed None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:52.763096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.764106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.764398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.764708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.778221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:52.784281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:52.785416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:52.786342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 388/797] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:52.787015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.826668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8c95f-3f83-4e4e-88c7-0337bf4871d4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:52.857618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:52.865857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612127ad-8bbe-41cf-89d9-f0d66105939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.867066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612127ad-8bbe-41cf-89d9-f0d66105939a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.867509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612127ad-8bbe-41cf-89d9-f0d66105939a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.867786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612127ad-8bbe-41cf-89d9-f0d66105939a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.879235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612127ad-8bbe-41cf-89d9-f0d66105939a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:52.880703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612127ad-8bbe-41cf-89d9-f0d66105939a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:52.889313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46975f38-161a-488b-bb15-1e2e778261d7 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-612127ad-8bbe-41cf-89d9-f0d66105939a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:52.894965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:52.895858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:52.896782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:52.896782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:52.896782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:52.896782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:52.896782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 410/798] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:52.897621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b249eb-fd73-4386-93d8-723767c99a08 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.897902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b249eb-fd73-4386-93d8-723767c99a08 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.898175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b249eb-fd73-4386-93d8-723767c99a08 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.903296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 411/799] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:52.904012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b249eb-fd73-4386-93d8-723767c99a08 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.930937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:52.931583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:52.931874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:52.932155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:52.939069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:52.940508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:52.952193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:52.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:52.954226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 389/800] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:52.954690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:52.969587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.022511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b249eb-fd73-4386-93d8-723767c99a08 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.061092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f5b15-1673-4a41-934e-64aedafab33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.061726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5b15-1673-4a41-934e-64aedafab33f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.062016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5b15-1673-4a41-934e-64aedafab33f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.062517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5b15-1673-4a41-934e-64aedafab33f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.063005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f5b15-1673-4a41-934e-64aedafab33f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:53.063318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f5b15-1673-4a41-934e-64aedafab33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.064710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f5b15-1673-4a41-934e-64aedafab33f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.065743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03938530-cbc6-4eac-9956-c4d87da5499f None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.073873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 412/801] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:53.075094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97527b73-6ca5-467d-b73c-6c4294834c33 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.075391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97527b73-6ca5-467d-b73c-6c4294834c33 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.075672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97527b73-6ca5-467d-b73c-6c4294834c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.094282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-97527b73-6ca5-467d-b73c-6c4294834c33 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:53.095476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-97527b73-6ca5-467d-b73c-6c4294834c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:53.104656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.111009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.111782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.112064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.112398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.113207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:53.113510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.115072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.129488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 390/802] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:53.130439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.171546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 413/803] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:13:53.172811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97527b73-6ca5-467d-b73c-6c4294834c33 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.232372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.248812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33677a99-8450-4d9f-8817-eb15e321d33b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.285438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.286067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.286350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.286625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.287362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:53.287661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.287875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97527b73-6ca5-467d-b73c-6c4294834c33 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.289126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.301507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 391/804] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:53.302317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.324397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.325066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.325342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.325612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.336298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.337760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:53.352686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:53.353709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:53.355007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:53.355007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:53.355007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:53.355007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:53.355007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 414/805] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:53.355007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.355702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.405578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ee775-484e-4a68-b0cc-ff7c4cb97309 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.440279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3ce3a-0d4d-498e-ac11-8cdb5fa7f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.441470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3ce3a-0d4d-498e-ac11-8cdb5fa7f922 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.441769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3ce3a-0d4d-498e-ac11-8cdb5fa7f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.442155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3ce3a-0d4d-498e-ac11-8cdb5fa7f922 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.449881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.450386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3ce3a-0d4d-498e-ac11-8cdb5fa7f922 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.451806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3ce3a-0d4d-498e-ac11-8cdb5fa7f922 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.453742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4b87d-0cca-401c-990a-f52a4b0f24ed None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-17d3ce3a-0d4d-498e-ac11-8cdb5fa7f922 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:53.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:53.464889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 392/806] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bbed4-0622-456f-9b0b-15a12eb2cb8f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.470552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bbed4-0622-456f-9b0b-15a12eb2cb8f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.471256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bbed4-0622-456f-9b0b-15a12eb2cb8f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.471256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 393/807] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:53.471256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bbed4-0622-456f-9b0b-15a12eb2cb8f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.493325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.493974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.494269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.494535 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.495154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.496517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:53.506006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:53.507124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 06:13:53.508166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:53.509181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 415/808] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:53.509900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.570197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bbed4-0622-456f-9b0b-15a12eb2cb8f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.583925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.606000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.606724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.607110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.607490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/a4141b562f1442efaa032122666724df/roles/b8e60301309b44f4acfea6244f79788b` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.610123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e73a-0f7e-4417-9a13-9a48ceb08e80 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.630644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=a4141b562f1442efaa032122666724df, role_id=b8e60301309b44f4acfea6244f79788b)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.632071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.643613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3dc65-4f88-4218-b5c2-221f820e5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.644301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3dc65-4f88-4218-b5c2-221f820e5abf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.644615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3dc65-4f88-4218-b5c2-221f820e5abf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.645400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3dc65-4f88-4218-b5c2-221f820e5abf None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.646088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3dc65-4f88-4218-b5c2-221f820e5abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:53.646521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3dc65-4f88-4218-b5c2-221f820e5abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.647930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3dc65-4f88-4218-b5c2-221f820e5abf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.652001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 394/809] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:53 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/a4141b562f1442efaa032122666724df/roles/b8e60301309b44f4acfea6244f79788b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:13:53.664851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.665010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 416/810] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:53.665010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c6a5b-de49-4f2a-b291-cf6beb168d96 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.665010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c6a5b-de49-4f2a-b291-cf6beb168d96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.665010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c6a5b-de49-4f2a-b291-cf6beb168d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.685883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7b3c6a5b-de49-4f2a-b291-cf6beb168d96 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:53.687128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7b3c6a5b-de49-4f2a-b291-cf6beb168d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:53.707138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.776971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 417/811] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:13:53.778476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c6a5b-de49-4f2a-b291-cf6beb168d96 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.796961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73038-aa84-4cd0-aa24-aa299c1621d3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.840196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.840861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.841144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.841421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.842369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:53.842682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.844129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.863621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 395/812] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:53.864529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.888099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:53.905499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c6a5b-de49-4f2a-b291-cf6beb168d96 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.947510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:53.948130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:53.948413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:53.948836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:53.949631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:53.949944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:53.951317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:53.961716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 418/813] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:53.961716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:53.974622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7352d70-a00c-49b3-a217-aabeea8ce6cd None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:53.981300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.010735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.011381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.011701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.011934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.020071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.021521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:54.033128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:54.034031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 396/814] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:54.035552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.073841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21784099-05e3-40f5-bed6-caa1dcb5666a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.110324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.110982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.111264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.111544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] PATH_INFO: `/v3/services/network` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.112177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.113934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:54.122920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:54.123922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 06:13:54.124770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:54.126803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.129506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 419/815] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:53 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:54.130418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.149441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942e5a-4778-45cb-bc83-8f0ec0ac4b08 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.186143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.186788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.187069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.187341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.198517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.199975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.215372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:54.217375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:54.218565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:54.219442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:54.219442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:54.219442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:54.219442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:54.219442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 397/816] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:54.220114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.242300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3052aa7-57d0-48cc-a159-b2abb878ec31 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.280256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.280956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.281243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.281523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.282204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:54.282509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.283877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.289813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 420/817] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:54.290602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.335647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e4d75-a0f1-4bed-8f91-4bd076a715f5 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.367121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.373001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.373754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.374041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.374326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.375113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:54.375426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.377105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.396555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13b23c-48b7-46a2-ae7d-b06fa19816a5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.402144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 398/818] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:54.402144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.437380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.438017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.438289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.439193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.439307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] RBAC: Adding query filter params (name=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:54.439556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.441591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.450873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 421/819] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:54.450873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.458617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.514637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac70305-6a95-41a7-99d1-a7008d18d7aa None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.552938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.553707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.554119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.554288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd903-ff73-4beb-9faf-72e851310078 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.554575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.555395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:54.555894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.558040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.570531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 399/820] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:54.570531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.594532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.595159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.595444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.595720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.596416 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.598148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.619559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 422/821] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:13:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:54.620712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.632696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.698324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb58ad-d947-4b8c-ab69-a1075a68183c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.733799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.752670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.753901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.754186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.754530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.786789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c9a9-9f9a-4952-8060-f10d7f758b0e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.830342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:54.831780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:54.831911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.832079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:54.832350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:54.833142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:54.833457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.833557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:54.834827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:54.841165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 423/822] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:54.842170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:54.900713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:54.969345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb6ba-6c3f-4b94-9460-f4079ef777a5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:54.974608 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 400/823] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:13:54.975596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.008067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:55.020894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.022192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.022912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.023337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.032583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.035006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:55.052306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:55.053422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:55.054508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:55.056565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 424/824] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:55.057557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.088813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e078f3-0c4a-4dd9-93c5-0f0b4cf63088 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.126582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.127402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.127730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.128138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.129084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.130539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.157927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 401/825] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:13:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:13:55.159041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.163524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:55.196770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3a4eb-b386-4297-88e9-b06da48d43d7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.244255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.245461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.245992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.246527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.260938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.263042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:55.280368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:55.281348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:55.282242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:55.282242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:55.282242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:55.282242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:55.285146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 425/826] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:55.290974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.294233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5f7ab-e528-4724-914d-b1815b79324c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.330816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:55.341623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.342216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.342515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.342832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.343600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:55.343913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.345399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.361684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 402/827] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:55.368329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.465683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997b2cf-72f2-4b1f-a34c-424380486ca7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.478623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:55.525711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.526688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.527365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.527789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.529346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:55.529802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.532257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.536209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b629d-42c8-446f-a283-a2aeec47c19c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.551356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 426/828] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:55.556967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.584068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1de9d-78c8-4839-ae1d-5f867017a092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.584946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1de9d-78c8-4839-ae1d-5f867017a092 None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.587548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1de9d-78c8-4839-ae1d-5f867017a092 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.587548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1de9d-78c8-4839-ae1d-5f867017a092 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/e35b790a4a5b4306a5537984d0b449e6/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.588310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1de9d-78c8-4839-ae1d-5f867017a092 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=e35b790a4a5b4306a5537984d0b449e6, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.590586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1de9d-78c8-4839-ae1d-5f867017a092 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.609547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 403/829] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:13:55 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/e35b790a4a5b4306a5537984d0b449e6/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:13:55.687226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:55.694523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7733-99a5-4a32-8f07-1ec5d81a9538 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.694725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af556b-1f26-4b21-a76c-0401eeb2c66e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.695063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7733-99a5-4a32-8f07-1ec5d81a9538 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.695675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7733-99a5-4a32-8f07-1ec5d81a9538 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.697229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 404/830] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:13:55.703332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fca78-0e40-4b7c-8847-7310e2b0d14e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.703899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fca78-0e40-4b7c-8847-7310e2b0d14e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.704353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fca78-0e40-4b7c-8847-7310e2b0d14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.728158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-750fca78-0e40-4b7c-8847-7310e2b0d14e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:55.729696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-750fca78-0e40-4b7c-8847-7310e2b0d14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:55.750548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.751215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.751501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.751778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.762546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.764788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:55.783794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:55.785833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:55.786910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 427/831] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:55.791378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.817115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 405/832] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:13:55.845904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fca78-0e40-4b7c-8847-7310e2b0d14e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.900904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:55.916082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffeae-5d11-44f9-b1f4-5ffe00ad857e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.949295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fca78-0e40-4b7c-8847-7310e2b0d14e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:55.953869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.955788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.956173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.956694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:55.957521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:55.957896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:55.959526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:55.974191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 428/833] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:55.980356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:55.988871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:55.989542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:55.989826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:55.990100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.002100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:56.004916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:56.012759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:56.013892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:13:56.014913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:56.022222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 406/834] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:13:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:56.023485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:56.093473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6528c85-e709-4137-8c02-44359b8f9a4c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:56.131313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa84572-4347-4a96-9e48-4bd2dc2b43b3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:56.132657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:56.133448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.133803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.134153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.135142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bbe64416672b408f8cdf93220cc31ef5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:56.135459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:56.136983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:56.148946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:56.149353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 429/835] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:55 2022] GET /identity/v3/role_assignments?user.id=bbe64416672b408f8cdf93220cc31ef5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:56.171040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:56.171973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.172402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.172853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.173751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:56.174208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:56.176344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:56.186439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 407/836] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:13:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:56.192884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:56.288943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:56.309401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bbbe5-472a-49d7-bb0a-4163eb0da6df None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:56.346467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:56.347189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.347628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.347921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.359069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:56.360524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:56.375967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:56.376946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:56.377811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:56.377811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:56.377811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:56.377811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:56.377811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 430/837] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:56.384318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:56.490060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23f8ee-7f79-4600-9cf2-47670cb6e94d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:56.525021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:56.525918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.526345 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.526763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.527592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:56.528046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:56.529580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:56.542219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 408/838] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:56.547842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:56.652999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca406007-7b50-47de-bb3e-7f541a606f22 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:56.684737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:56.691952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:56.692871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.693296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.693712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.701885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:56.703478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:56.715398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:56.716553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:13:56.717706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:56.719892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 431/839] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:13:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:56.725809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:56.763258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:56.832477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713ccc0-7b32-4eff-a0aa-bb83f7047398 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:56.870548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a623447-9fde-4f63-94df-a7d8310b6618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:56.871458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a623447-9fde-4f63-94df-a7d8310b6618 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.871886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a623447-9fde-4f63-94df-a7d8310b6618 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.872327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a623447-9fde-4f63-94df-a7d8310b6618 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.873284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a623447-9fde-4f63-94df-a7d8310b6618 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:56.873734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a623447-9fde-4f63-94df-a7d8310b6618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:56.875291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a623447-9fde-4f63-94df-a7d8310b6618 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:56.894206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 409/840] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 29 06:13:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:56.900157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:56.908753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fbbf6-8bbe-4d43-9212-1e91887dd20f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.909050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fbbf6-8bbe-4d43-9212-1e91887dd20f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.909380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fbbf6-8bbe-4d43-9212-1e91887dd20f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.910966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 410/841] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:56.915821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39418991-3d6c-42c4-96e5-40b9f89aa214 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:56.916116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39418991-3d6c-42c4-96e5-40b9f89aa214 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:56.916401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39418991-3d6c-42c4-96e5-40b9f89aa214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:56.935099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-39418991-3d6c-42c4-96e5-40b9f89aa214 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:56.936289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-39418991-3d6c-42c4-96e5-40b9f89aa214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:57.013875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70bcd2-d33d-4276-97cb-881bf8bb6082 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:57.021090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 411/842] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:13:57.042738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39418991-3d6c-42c4-96e5-40b9f89aa214 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.052553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:57.053284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.053567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.053849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.068082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:57.118295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:57.131323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:57.133152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:57.150413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39418991-3d6c-42c4-96e5-40b9f89aa214 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:57.187607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:57.188263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.188565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.188868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.189502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:57.191571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:57.201035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:57.202226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:13:57.203050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:57.208310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 412/843] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:57.209231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.245348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 432/844] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:13:57.255153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.298842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:57.319120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44acdd-5514-4a07-a323-52147b0fdbc0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:57.360464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:57.366996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:57.367633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.367920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.368126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ef252-d17d-44e8-a4ee-c1dc50992d60 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:57.368236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.368953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:57.369277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:57.371871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:57.380629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 413/845] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:57.383908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.422625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:57.423287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.423574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.423856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.435583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:57.437162 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:57.453561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:57.454490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:57.455418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:57.455418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:57.455418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:57.455418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:57.460116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 433/846] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:57.461418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.515013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdb84a-caf1-4bfe-9d32-3d6af03d54cb None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:57.560657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:57.561818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.562609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.563119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.564234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:57.564936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:57.567190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:57.571530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc53c-9be7-4e60-81fb-05348038c43f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:57.581791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 414/847] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:57.618537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c5812-db97-4072-9d90-bceabb72e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:57.619240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5812-db97-4072-9d90-bceabb72e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.619467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5812-db97-4072-9d90-bceabb72e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.619746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5812-db97-4072-9d90-bceabb72e1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.620446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c5812-db97-4072-9d90-bceabb72e1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:57.620798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c5812-db97-4072-9d90-bceabb72e1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:57.622246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c5812-db97-4072-9d90-bceabb72e1a0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:57.640717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 434/848] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:57.642677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.661793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:57.762898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e57b0c-f7e5-4921-884e-d130a05adeb1 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:57.802026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:57.802674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.802960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.803238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.810580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:57.812043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:57.822407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780391b-4b09-4696-8b81-97bfb9f2593e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.822694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780391b-4b09-4696-8b81-97bfb9f2593e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.822974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780391b-4b09-4696-8b81-97bfb9f2593e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:13:57.826583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:57.827461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:13:57.828339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:57.828806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 435/849] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:57.828876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 415/850] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:57.830759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8879a9-171b-4aa9-8f02-c15100153b24 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:57.831085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8879a9-171b-4aa9-8f02-c15100153b24 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:57.831422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8879a9-171b-4aa9-8f02-c15100153b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:57.843005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.843162 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:57.860217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7f8879a9-171b-4aa9-8f02-c15100153b24 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:57.861930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7f8879a9-171b-4aa9-8f02-c15100153b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:57.944347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:57.953771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 436/851] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:13:57.977568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8879a9-171b-4aa9-8f02-c15100153b24 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:57.979239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad3fd0-3d10-43fb-806a-562bbaef1fe0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.015035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.016235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.016235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.016435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.017067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:58.017377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.019603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.033055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 416/852] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:58.039452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.092665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8879a9-171b-4aa9-8f02-c15100153b24 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.134740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.135794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.136404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.136931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.139301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:58.149614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd95ff2-dd9a-44b8-86ab-0af8ffcde322 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.151746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:13:58.157110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:58.158046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:13:58.158916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:58.162540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 437/853] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:58.174268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.192335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff92f76-d775-4ea6-b5d4-58ee1b7fce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.193020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff92f76-d775-4ea6-b5d4-58ee1b7fce25 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.193298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff92f76-d775-4ea6-b5d4-58ee1b7fce25 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.193653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff92f76-d775-4ea6-b5d4-58ee1b7fce25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.194417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff92f76-d775-4ea6-b5d4-58ee1b7fce25 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=a4141b562f1442efaa032122666724df) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:58.194674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff92f76-d775-4ea6-b5d4-58ee1b7fce25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.196110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff92f76-d775-4ea6-b5d4-58ee1b7fce25 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.198122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:58.211431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 417/854] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/role_assignments?user.id=a4141b562f1442efaa032122666724df&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:58.262318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dda6a-881d-439a-8678-02cd51c63245 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.262676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dda6a-881d-439a-8678-02cd51c63245 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.262876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dda6a-881d-439a-8678-02cd51c63245 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.268912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 418/855] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:58.270071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a31ec3-9c79-49e7-bcb7-bded7ae263e5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.270356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a31ec3-9c79-49e7-bcb7-bded7ae263e5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.270631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a31ec3-9c79-49e7-bcb7-bded7ae263e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.295214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-67a31ec3-9c79-49e7-bcb7-bded7ae263e5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:58.297156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-67a31ec3-9c79-49e7-bcb7-bded7ae263e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:58.305201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af413f88-9b0c-46ed-804b-d67c0a594d57 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.344677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.345451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.345737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.346014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.347616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:58.347780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.349270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.359619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 438/856] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:58.366008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.399303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 419/857] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:13:58.421697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a31ec3-9c79-49e7-bcb7-bded7ae263e5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.433973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:58.495175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867634b-40a5-45ae-bd04-8e3f4c49ae50 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.532072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.532785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.533058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.533369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.545434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.545863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a31ec3-9c79-49e7-bcb7-bded7ae263e5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.546888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:58.562942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:58.563884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:58.565506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:58.565506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:58.565506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:58.565506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:58.565506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 439/858] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:58.570271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.586357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:58.606645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.607483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.607766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.608043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.608960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.610438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:58.620860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:58.621833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:13:58.622690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 420/859] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:13:58.629003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.698553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b00d6-3ea8-47f2-89dc-0533e1c7a4de None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.709651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:58.738569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.739524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.739970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.740370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.741296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:58.741711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.743249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.756986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44969219-6461-4fb8-ad0a-fe9ca8f6d8cb None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.761764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 440/860] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:13:58.762823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.795026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.796021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.796477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.796930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.797807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:58.798277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.800170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.810731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 421/861] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:58.811950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.873310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:58.879719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc2d82-77bf-4011-b323-bd8fd75ab14f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.918908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e5789-f475-4708-a339-01553947e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.919780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e5789-f475-4708-a339-01553947e36f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.920177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e5789-f475-4708-a339-01553947e36f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.920575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e5789-f475-4708-a339-01553947e36f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.921721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91acd5-067d-4039-aec9-ec95865a5258 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:58.928813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e5789-f475-4708-a339-01553947e36f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.930494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e5789-f475-4708-a339-01553947e36f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-790e5789-f475-4708-a339-01553947e36f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:13:58.943860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:58.944956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 06:13:58.945887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:58.950211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 441/862] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:58.951317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e5789-f475-4708-a339-01553947e36f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:58.960994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:58.961855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:58.962313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:58.962759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:58.963596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:58.964097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:58.965657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:58.976677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 422/863] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:59.005015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:59.071294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e5789-f475-4708-a339-01553947e36f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:59.110521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:59.111196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.111427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.111711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:59.112618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:59.112910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:59.114372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:59.134820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 442/864] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 29 06:13:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:59.140451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:59.257404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f5a10-ca6a-4124-8073-944beeb6fdfa None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:59.278908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:59.295944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:59.296918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.297410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.297878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:59.372064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:59.373548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:59.484197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 423/865] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:13:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:13:59.490350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:59.581958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:59.598247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9311e-4925-49e8-83ce-875ec71c76ef None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:59.633071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435107ac-c883-41c9-970e-2b06c7bcd189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:59.633648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435107ac-c883-41c9-970e-2b06c7bcd189 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.633934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435107ac-c883-41c9-970e-2b06c7bcd189 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.634264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435107ac-c883-41c9-970e-2b06c7bcd189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:59.644918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435107ac-c883-41c9-970e-2b06c7bcd189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:59.646369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435107ac-c883-41c9-970e-2b06c7bcd189 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-435107ac-c883-41c9-970e-2b06c7bcd189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:13:59.662767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:13:59.663813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:13:59.664863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:13:59.664863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:13:59.664863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:13:59.664863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:13:59.664863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 443/866] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:13:59.670002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:59.692402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:59.711465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcbadd-835a-42b5-836e-8ce034d9d193 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.711743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcbadd-835a-42b5-836e-8ce034d9d193 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.712025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcbadd-835a-42b5-836e-8ce034d9d193 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:59.713674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 444/867] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:13:59.718390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081ce4e-e071-4efe-9553-632d71715599 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.718675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081ce4e-e071-4efe-9553-632d71715599 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.718954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081ce4e-e071-4efe-9553-632d71715599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:59.736714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3081ce4e-e071-4efe-9553-632d71715599 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:13:59.738439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3081ce4e-e071-4efe-9553-632d71715599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:13:59.786127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f5fb-800a-477b-a47d-43546717b8fe None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:59.816624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 445/868] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:13:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:13:59.822558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:59.823260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.823617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.823967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:59.824760 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:13:59.825136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:59.826569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:59.831020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081ce4e-e071-4efe-9553-632d71715599 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:59.839635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 424/869] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:13:59.845209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:13:59.899030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:59.934609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:13:59.936110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081ce4e-e071-4efe-9553-632d71715599 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:59.956877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e408a-267e-4eeb-9a30-d952a3004940 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:13:59.982666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:59.983831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.984748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.985473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:13:59.986624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:13:59.989488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:13:59.995738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:13:59.996372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:13:59.996752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:13:59.996977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:00.005448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:00.006936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:00.009446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 446/870] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:13:59 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:00.024003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:00.025045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:00.025950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:00.028416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 425/871] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:00.035114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:00.167838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a9f77-1122-4978-b138-bcc8d77fa2f6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:00.197140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:00.213006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d809a-ae0a-4b85-8f4e-969c9445e9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:00.213926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d809a-ae0a-4b85-8f4e-969c9445e9be None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:00.214388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d809a-ae0a-4b85-8f4e-969c9445e9be None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:00.214816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d809a-ae0a-4b85-8f4e-969c9445e9be None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:00.215721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d809a-ae0a-4b85-8f4e-969c9445e9be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:00.216181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d809a-ae0a-4b85-8f4e-969c9445e9be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:00.217806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d809a-ae0a-4b85-8f4e-969c9445e9be None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:00.234048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 447/872] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:00.240633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:00.351290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf7063-ac8d-42ad-9e1d-a90862f9c6cd None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:00.388094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d8805-597f-4a2d-bd6a-9d63e8382021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:00.388991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8805-597f-4a2d-bd6a-9d63e8382021 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:00.389459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8805-597f-4a2d-bd6a-9d63e8382021 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:00.389896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8805-597f-4a2d-bd6a-9d63e8382021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:00.390882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d8805-597f-4a2d-bd6a-9d63e8382021 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=e35b790a4a5b4306a5537984d0b449e6) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:00.391344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d8805-597f-4a2d-bd6a-9d63e8382021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:00.392941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d8805-597f-4a2d-bd6a-9d63e8382021 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:00.399702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 426/873] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:00 2022] GET /identity/v3/role_assignments?user.id=e35b790a4a5b4306a5537984d0b449e6&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:00.473016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa151f5-c85c-4f6c-a9c2-63ced041bb27 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:00.473585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa151f5-c85c-4f6c-a9c2-63ced041bb27 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:00.474051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa151f5-c85c-4f6c-a9c2-63ced041bb27 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:00.475923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 448/874] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:00.481323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8cda9-1a01-443d-874b-de75d489d00a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:00.481980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8cda9-1a01-443d-874b-de75d489d00a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:00.482444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8cda9-1a01-443d-874b-de75d489d00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:00.491296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:00.505304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e5c8cda9-1a01-443d-874b-de75d489d00a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:00.506903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e5c8cda9-1a01-443d-874b-de75d489d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:00.582723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 427/875] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:00.607128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa151f5-c85c-4f6c-a9c2-63ced041bb27 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:00.713617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa151f5-c85c-4f6c-a9c2-63ced041bb27 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:00.751777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:00.752381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:00.752698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:00.753012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:00.763975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:00.769066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:00.769983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:00.770962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 449/876] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 29 06:14:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:00.778159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8cda9-1a01-443d-874b-de75d489d00a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:00.886483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8cda9-1a01-443d-874b-de75d489d00a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:00.923971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:00.924616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:00.924933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:00.925206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:00.925919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:00.926220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:00.927582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:00.928946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:00.936158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 428/877] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 29 06:14:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:00.941901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:00.997968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:01.063044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77f7b4-eb6c-423d-b962-b83b26cc4b04 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:01.108584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31028b-1403-4fd6-8737-3f3eeec84478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:01.109588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31028b-1403-4fd6-8737-3f3eeec84478 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.110076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31028b-1403-4fd6-8737-3f3eeec84478 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.110554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31028b-1403-4fd6-8737-3f3eeec84478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.121422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31028b-1403-4fd6-8737-3f3eeec84478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:01.123172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31028b-1403-4fd6-8737-3f3eeec84478 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ee31028b-1403-4fd6-8737-3f3eeec84478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:01.138709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:01.139616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:01.140527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:01.140527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:01.140527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:01.140527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:01.147902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 450/878] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:01.149537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:01.185822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffddb2-418e-440f-b915-32e9ca1473cb None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.186770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffddb2-418e-440f-b915-32e9ca1473cb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.187251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffddb2-418e-440f-b915-32e9ca1473cb None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.189340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 451/879] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:01.195778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cbdd-5934-454f-a5a1-0016dead0d23 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.196337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cbdd-5934-454f-a5a1-0016dead0d23 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.196838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153cbdd-5934-454f-a5a1-0016dead0d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.218955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e153cbdd-5934-454f-a5a1-0016dead0d23 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:01.220862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e153cbdd-5934-454f-a5a1-0016dead0d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:01.273290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8429a-c76e-4182-a856-4b87586967f3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:01.313314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:01.313940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.314221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.314494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.315179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:01.316026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:01.317411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:01.320622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 452/880] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:01.332483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 429/881] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:01.338004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153cbdd-5934-454f-a5a1-0016dead0d23 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:01.341627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:01.371267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:01.421285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:01.458784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153cbdd-5934-454f-a5a1-0016dead0d23 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:01.487133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68516d-35bf-4f36-890a-e475c71e81e8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:01.495621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:01.496398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.497041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.497358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.505816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:01.507134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:14:01.519482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:01.520673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:14:01.521887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:01.527632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 453/882] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:14:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:01.529469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:01.535932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:01.537522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.537821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.538103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.538804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:01.540178 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:01.564697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 430/883] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:14:01 2022] POST /identity/v3/services => generated 269 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:01.641613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:01.659419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1399a-a5d0-4edb-8304-998f0d32af03 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:01.704167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:01.704858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.705146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.705425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.706480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:01.706796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:01.708351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:01.744827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 454/884] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 29 06:14:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:01.750659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:01.864129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c477e-1cbd-462a-aa77-a3c12a5df929 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:01.881678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:01.901774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01907348-9f9e-4da1-97eb-d42ff415ab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:01.902414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907348-9f9e-4da1-97eb-d42ff415ab5f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:01.902707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907348-9f9e-4da1-97eb-d42ff415ab5f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:01.902993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907348-9f9e-4da1-97eb-d42ff415ab5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:01.974832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01907348-9f9e-4da1-97eb-d42ff415ab5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:01.976285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01907348-9f9e-4da1-97eb-d42ff415ab5f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.077862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 431/885] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:02.084477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.193726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d77acd-7640-42bc-a571-54e213e23812 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:02.199629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53839f62-a50e-439f-b32e-c3bc01b68e5b None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.199971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53839f62-a50e-439f-b32e-c3bc01b68e5b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.200210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53839f62-a50e-439f-b32e-c3bc01b68e5b None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.201870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 432/886] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:02.207679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33fcf-ef44-4dc9-a808-2171c135633b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.207970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33fcf-ef44-4dc9-a808-2171c135633b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.208252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33fcf-ef44-4dc9-a808-2171c135633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.227003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-6de33fcf-ef44-4dc9-a808-2171c135633b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:02.228212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6de33fcf-ef44-4dc9-a808-2171c135633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:02.232989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:02.233638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.233919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.234195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.240376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:02.246975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:02.251298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:02.253310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:02.272163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:02.273420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:02.275410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:02.275410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:02.275410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:02.275410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:02.275410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 455/887] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:02.281360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.311250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 433/888] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:02.335010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de33fcf-ef44-4dc9-a808-2171c135633b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.426979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d7451-9f03-4fee-9d23-99094845e277 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:02.446568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de33fcf-ef44-4dc9-a808-2171c135633b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:02.481257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:02.482248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.482735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.483233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.483912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:02.484375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:02.484650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.484939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.485217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.485564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:02.485843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:02.487862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.488391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:02.497858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:02.498849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:14:02.499669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:02.504920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 434/889] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:02.505844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.511747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 456/890] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:02.514144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.524487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:02.573867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:02.625439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78b5e-4f74-4675-a14d-3297157c0a75 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:02.638986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b8e3d-f1d8-466c-9fbd-c17db37baad7 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:02.662665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f964d-5e5f-4fa5-ac8e-a6e2383e1c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:02.663307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f964d-5e5f-4fa5-ac8e-a6e2383e1c2f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.663587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f964d-5e5f-4fa5-ac8e-a6e2383e1c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.663859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f964d-5e5f-4fa5-ac8e-a6e2383e1c2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.670476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f964d-5e5f-4fa5-ac8e-a6e2383e1c2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:02.671929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f964d-5e5f-4fa5-ac8e-a6e2383e1c2f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.676855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f39ce1-865b-4c9c-9ef4-3045a834e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:02.677481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39ce1-865b-4c9c-9ef4-3045a834e762 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.677768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39ce1-865b-4c9c-9ef4-3045a834e762 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.678049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39ce1-865b-4c9c-9ef4-3045a834e762 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.678721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39ce1-865b-4c9c-9ef4-3045a834e762 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:02.679028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39ce1-865b-4c9c-9ef4-3045a834e762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:02.680385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39ce1-865b-4c9c-9ef4-3045a834e762 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-952f964d-5e5f-4fa5-ac8e-a6e2383e1c2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:02.683101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:02.683964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 457/891] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:02.685515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78878ae-a332-46aa-ada2-c1b5a7fbadda None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.686054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78878ae-a332-46aa-ada2-c1b5a7fbadda None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.686054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78878ae-a332-46aa-ada2-c1b5a7fbadda None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.687332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 458/892] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:02.691181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 435/893] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:02.691914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78878ae-a332-46aa-ada2-c1b5a7fbadda None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.693990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eeb33-dc4c-4a2a-95e7-6aa268463702 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.694795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eeb33-dc4c-4a2a-95e7-6aa268463702 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.696492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eeb33-dc4c-4a2a-95e7-6aa268463702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.716103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ee1eeb33-dc4c-4a2a-95e7-6aa268463702 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:02.720082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ee1eeb33-dc4c-4a2a-95e7-6aa268463702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:02.795831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 436/894] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:02.796949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eeb33-dc4c-4a2a-95e7-6aa268463702 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.801365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:02.826770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78878ae-a332-46aa-ada2-c1b5a7fbadda None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:02.844684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:02.868728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:02.869358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.869639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.870085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.871422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:02.871752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:02.873221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.891187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 459/895] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:02.891187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.912900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eeb33-dc4c-4a2a-95e7-6aa268463702 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:02.949152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:02.949777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:02.950071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:02.950353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:02.951030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:02.951340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:02.952815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:02.963337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 437/896] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:02.963337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:02.996344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26db2d-fe07-43ba-a8b5-61e33e24638b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:03.030915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:03.031564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.031851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.032132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.032789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:03.034243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:03.038303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:03.043750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:03.045089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:03.045962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:03.050152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 460/897] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:03.050837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:03.075302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d44fbe-b03c-4d55-8b19-c2567e725d31 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:03.087604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:03.115518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:03.116245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.116558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.116846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.117619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] RBAC: Adding query filter params (role.id=ab91e6fc0e6942c2bd23a93741d5b1d8, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=a4141b562f1442efaa032122666724df) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:03.117930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:03.119350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:03.125434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 438/898] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:02 2022] GET /identity/v3/role_assignments?user.id=a4141b562f1442efaa032122666724df&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=ab91e6fc0e6942c2bd23a93741d5b1d8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:03.126240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:03.171516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed034335-e9da-46ea-ad49-6190bcea5ffa None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:03.215083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:03.215753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.216039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.216258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.217058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] RBAC: Adding query filter params (interface=public, service_id=e4849ccf6e0e4fdab9980d442617dac3, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:03.217372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:03.219000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:03.230762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 461/899] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:14:03 2022] GET /identity/v3/endpoints?service_id=e4849ccf6e0e4fdab9980d442617dac3&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:03.232239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:03.248683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710da0d-908f-4a3f-ab5b-3bb839e92451 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:03.282080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:03.289621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:03.290271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.290553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.290828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.291505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:03.291811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:03.294349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:03.300529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 439/900] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:03.308446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:03.371086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:03.374663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed1e0-eff9-4110-b854-097c4a88fd23 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:03.423164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d614d10-0747-4e70-a548-483c8e43dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:03.423952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d614d10-0747-4e70-a548-483c8e43dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.424341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d614d10-0747-4e70-a548-483c8e43dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.425673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d614d10-0747-4e70-a548-483c8e43dfc6 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.427213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d614d10-0747-4e70-a548-483c8e43dfc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:03.429193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d614d10-0747-4e70-a548-483c8e43dfc6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:03.432984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751805a-9748-4b2f-8dea-497bda7d79a7 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:03.437069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 462/901] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:14:03 2022] GET /identity/v3/services => generated 1268 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:14:03.474388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6dd09e-d1e7-4178-861f-18c93eb2c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:03.475356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dd09e-d1e7-4178-861f-18c93eb2c78f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.475845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dd09e-d1e7-4178-861f-18c93eb2c78f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.476306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dd09e-d1e7-4178-861f-18c93eb2c78f None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.477335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6dd09e-d1e7-4178-861f-18c93eb2c78f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:03.477835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6dd09e-d1e7-4178-861f-18c93eb2c78f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:03.479526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6dd09e-d1e7-4178-861f-18c93eb2c78f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:03.485342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 440/902] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:03.689885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458300-2afc-4a2b-930f-89d6bfec4114 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.690297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458300-2afc-4a2b-930f-89d6bfec4114 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.690499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458300-2afc-4a2b-930f-89d6bfec4114 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.691968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 463/903] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:03.697341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ecf9a-67aa-4fc0-8fd0-416de35df30d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.697860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ecf9a-67aa-4fc0-8fd0-416de35df30d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.698335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ecf9a-67aa-4fc0-8fd0-416de35df30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.717390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-fc6ecf9a-67aa-4fc0-8fd0-416de35df30d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:03.718925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fc6ecf9a-67aa-4fc0-8fd0-416de35df30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:03.783405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:03.796669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 441/904] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:03.819249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76458300-2afc-4a2b-930f-89d6bfec4114 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:03.928695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76458300-2afc-4a2b-930f-89d6bfec4114 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:03.965490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:03.966059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:03.966343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:03.966618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:03.967248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:03.968640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:03.978244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:03.979187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:03.980008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 464/905] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jul 29 06:14:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:03.987556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ecf9a-67aa-4fc0-8fd0-416de35df30d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:04.091395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ecf9a-67aa-4fc0-8fd0-416de35df30d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:04.137030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:04.137664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:04.137953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:04.138231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:04.138914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:04.139222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:04.140600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:04.146473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 442/906] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:04.153692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:04.192940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:04.263608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2622876-d813-48af-85fd-978ce7fa86af None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:04.301065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:04.302078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:04.302560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:04.303051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:04.303919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:04.304436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:04.306062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:04.311460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 465/907] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:04.318470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:04.382540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:04.429599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c67d-a598-4687-b324-f1ee8f8948a3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:04.468957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3522b-e55d-41ed-b747-75602c2f2441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:04.470242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3522b-e55d-41ed-b747-75602c2f2441 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:04.470692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3522b-e55d-41ed-b747-75602c2f2441 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:04.471115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3522b-e55d-41ed-b747-75602c2f2441 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:04.472163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3522b-e55d-41ed-b747-75602c2f2441 None admin] RBAC: Adding query filter params (interface=public, service_id=31b8d745eab94b05bed8c81cd62e81f9, region_id=RegionOne) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:04.472680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3522b-e55d-41ed-b747-75602c2f2441 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:04.474182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3522b-e55d-41ed-b747-75602c2f2441 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:04.479866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 443/908] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:14:04 2022] GET /identity/v3/endpoints?service_id=31b8d745eab94b05bed8c81cd62e81f9&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:04.488571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:04.579073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:04.588950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221726-32b2-406b-83af-5115ac2d2a77 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:04.624606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dab5c-0d4c-4539-9731-b241f4c1262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:04.625401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dab5c-0d4c-4539-9731-b241f4c1262d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:04.625778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dab5c-0d4c-4539-9731-b241f4c1262d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:04.626154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dab5c-0d4c-4539-9731-b241f4c1262d None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:04.626957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3dab5c-0d4c-4539-9731-b241f4c1262d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:04.629248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3dab5c-0d4c-4539-9731-b241f4c1262d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:04.635554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 466/909] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:14:04 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:14:05.179928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c261e-bc1b-4667-a81c-d7e3b94a143a None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.180242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c261e-bc1b-4667-a81c-d7e3b94a143a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.180514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c261e-bc1b-4667-a81c-d7e3b94a143a None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.182168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 444/910] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:05.187341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6b1d-2f51-437b-bd9d-a8aa38be0c97 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.187633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6b1d-2f51-437b-bd9d-a8aa38be0c97 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.187912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6b1d-2f51-437b-bd9d-a8aa38be0c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.207410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4dce6b1d-2f51-437b-bd9d-a8aa38be0c97 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:05.208648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4dce6b1d-2f51-437b-bd9d-a8aa38be0c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:05.278545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 467/911] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:05.301444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c261e-bc1b-4667-a81c-d7e3b94a143a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:05.416881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c261e-bc1b-4667-a81c-d7e3b94a143a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:05.436744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:05.455472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f325d-f06c-4ce7-96eb-aea8167a352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:05.457306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f325d-f06c-4ce7-96eb-aea8167a352d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.457605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f325d-f06c-4ce7-96eb-aea8167a352d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.457895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f325d-f06c-4ce7-96eb-aea8167a352d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.470371 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f325d-f06c-4ce7-96eb-aea8167a352d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c38f325d-f06c-4ce7-96eb-aea8167a352d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:05.476568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:05.477495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:05.478375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:05.481100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 445/912] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 29 06:14:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:05.486659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce6b1d-2f51-437b-bd9d-a8aa38be0c97 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:05.573430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:05.592623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce6b1d-2f51-437b-bd9d-a8aa38be0c97 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:05.595378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bda17-a349-448b-8d3e-ef290a1135b2 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.595673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bda17-a349-448b-8d3e-ef290a1135b2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.595962 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bda17-a349-448b-8d3e-ef290a1135b2 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.597648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 446/913] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:05.602655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b577b-900d-4d4e-94b2-fade6c552ed9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.602950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b577b-900d-4d4e-94b2-fade6c552ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.603235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b577b-900d-4d4e-94b2-fade6c552ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.622183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c07b577b-900d-4d4e-94b2-fade6c552ed9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:05.623491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c07b577b-900d-4d4e-94b2-fade6c552ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:05.629249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33996a-fdd6-4d57-b91b-ccb77b45681f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:05.629879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33996a-fdd6-4d57-b91b-ccb77b45681f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.630217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33996a-fdd6-4d57-b91b-ccb77b45681f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.630493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33996a-fdd6-4d57-b91b-ccb77b45681f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.631246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33996a-fdd6-4d57-b91b-ccb77b45681f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:05.631553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33996a-fdd6-4d57-b91b-ccb77b45681f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:05.632944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33996a-fdd6-4d57-b91b-ccb77b45681f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:05.644737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 468/914] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 29 06:14:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:05.644737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b080f-912d-4cb7-9334-59ef02985af0 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.644737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b080f-912d-4cb7-9334-59ef02985af0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.644737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b080f-912d-4cb7-9334-59ef02985af0 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.644737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 469/915] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:05.647132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b080f-912d-4cb7-9334-59ef02985af0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:05.701382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 447/916] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:05.702655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615c7b-1d99-49f9-a4c2-c41135ba77cd None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.702950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615c7b-1d99-49f9-a4c2-c41135ba77cd None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.703232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615c7b-1d99-49f9-a4c2-c41135ba77cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.729808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8e615c7b-1d99-49f9-a4c2-c41135ba77cd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:05.733403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8e615c7b-1d99-49f9-a4c2-c41135ba77cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:05.771246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b080f-912d-4cb7-9334-59ef02985af0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:05.808488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 448/917] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:05.809354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e615c7b-1d99-49f9-a4c2-c41135ba77cd None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:05.826647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:05.827188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.827519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.827801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.839462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:05.840935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:05.849567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:05.855739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:05.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:05.857851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:05.858764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:05.858764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:05.858764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:05.858764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:05.866357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 470/918] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:05.866357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:05.925674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e615c7b-1d99-49f9-a4c2-c41135ba77cd None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:05.962245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:05.963414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:05.963789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:05.964123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:05.964776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:05.966282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:05.969274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac009c55-b155-4ade-a908-30ed49a1487d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:05.976136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:05.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:05.978017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 449/919] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:14:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:05.978811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.009308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.009929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.010213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.010492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.011300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.012802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.039332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 471/920] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:14:05 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:06.040412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.109682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a2f31-c531-4107-a1d0-bacb8f3d05b3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.112127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:06.119053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:06.176518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.177351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.178066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.178455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.179261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:06.179678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.180024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450849e-12ca-42a4-ad93-9093fd182202 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.181320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.197698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 450/921] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:06.203390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.220671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.222099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.222527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.222943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.223807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:06.224253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.225913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.231918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 472/922] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:14:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:06.239548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.326554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05b3ed-5494-4369-a953-c62f314f8064 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.371722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.372758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.373045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.373325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.379563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af3114-68b3-45bb-9e5e-06a4f916971c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.382424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.385069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.404487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:14:06.407301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:06.408330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:06.414595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 451/923] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:14:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:06.416852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.434630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.435833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.436877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.437633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.438785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:06.439333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.440039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:06.443384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.458345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 473/924] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 29 06:14:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:06.459845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.533276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925995d-d643-4b8e-a1e6-714dc229342f None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.568028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58f50-5bb9-43fc-8613-c55e8caad3a3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.573167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.573801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.574090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.574370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.575167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:06.575478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.576939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.601678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 452/925] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 29 06:14:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:06.603777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.612454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c189b77-f947-440a-ba62-21564767d2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.613129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c189b77-f947-440a-ba62-21564767d2ce None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.613419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c189b77-f947-440a-ba62-21564767d2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.613710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c189b77-f947-440a-ba62-21564767d2ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.614427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c189b77-f947-440a-ba62-21564767d2ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.616427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c189b77-f947-440a-ba62-21564767d2ce None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.652576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 474/926] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:14:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:14:06.701427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:06.716624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e997ac-3d94-4ccb-96e3-1c53bc723fd5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.716942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e997ac-3d94-4ccb-96e3-1c53bc723fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.717197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e997ac-3d94-4ccb-96e3-1c53bc723fd5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.718840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 475/927] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:06.727644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0381ae-ae52-4393-a863-e8430a6059eb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.727947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0381ae-ae52-4393-a863-e8430a6059eb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.728216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0381ae-ae52-4393-a863-e8430a6059eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.743996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64370-54c3-4067-9a8a-0dfc06d1088b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.751989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ec0381ae-ae52-4393-a863-e8430a6059eb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:06.753863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ec0381ae-ae52-4393-a863-e8430a6059eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:06.792194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:06.793244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:06.793746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:06.794216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:06.805266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:06.806931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:06.808761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:06.827119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:06.828044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:06.829269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:06.829269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:06.829269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:06.829269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:06.829269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 453/928] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:06.836986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.848320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 476/929] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:06.870002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0381ae-ae52-4393-a863-e8430a6059eb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:06.968676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac31a8-05a5-4a7a-8c55-3453a32f2458 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:06.985453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0381ae-ae52-4393-a863-e8430a6059eb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:07.000220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:07.016406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.017491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.018345 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.019002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.020071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:07.020459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.022791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.027149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.028013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.028479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.028981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.029789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.031450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:07.041700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:07.042743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 06:14:07.043601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:07.044385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 454/930] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:07.044385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:07.046139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 477/931] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jul 29 06:14:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:07.053479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:07.103015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:07.165709 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9dc50-452b-4bb6-a2c2-6be4e1f06918 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:07.185630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef063f1-cf4f-422d-b317-0511e2ed5c12 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:07.204163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.205282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.206611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.207329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.215732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.217681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.225624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.226271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.226558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.226843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.227533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:07.227846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.229265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.239260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 478/932] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:07.239434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:07.240277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:07.241205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:07.244291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:07.244618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 455/933] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:07.246244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:07.310251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:07.361556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:07.367669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52e360-8a0b-47dc-b8db-aa92f3043d29 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:07.390836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1f920-ca23-4107-a436-b2934010181e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:07.405858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.407138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.407640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.408133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.409143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:07.409693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.411281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.420931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 479/934] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:07.423035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:07.438634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.439269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.439557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.439835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.441214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:07.441687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.443128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.461645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 456/935] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:14:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:07.463871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:07.548087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9af5b-e302-4683-9671-208a2731a039 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:07.587640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a45bf7-bb97-4c16-986a-e0f61b3e69e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.588273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a45bf7-bb97-4c16-986a-e0f61b3e69e8 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.588593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a45bf7-bb97-4c16-986a-e0f61b3e69e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.588885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a45bf7-bb97-4c16-986a-e0f61b3e69e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.589691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a45bf7-bb97-4c16-986a-e0f61b3e69e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.591325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a45bf7-bb97-4c16-986a-e0f61b3e69e8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.602422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3a942-cb82-4b28-a95d-f04b6a5dba52 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:07.606554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:07.623300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 480/936] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:14:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:07.653070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722b7af-ed3c-44e2-958d-a0da388d23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:07.653772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722b7af-ed3c-44e2-958d-a0da388d23ae None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:07.654059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722b7af-ed3c-44e2-958d-a0da388d23ae None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:07.654342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722b7af-ed3c-44e2-958d-a0da388d23ae None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/a4141b562f1442efaa032122666724df/roles/ab91e6fc0e6942c2bd23a93741d5b1d8` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:07.656685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722b7af-ed3c-44e2-958d-a0da388d23ae None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=a4141b562f1442efaa032122666724df, role_id=ab91e6fc0e6942c2bd23a93741d5b1d8)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:07.658243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722b7af-ed3c-44e2-958d-a0da388d23ae None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:07.680786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 457/937] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:14:07 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/a4141b562f1442efaa032122666724df/roles/ab91e6fc0e6942c2bd23a93741d5b1d8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:14:08.129443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80989-1905-40fc-b946-44821c565031 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.129854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80989-1905-40fc-b946-44821c565031 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.130047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80989-1905-40fc-b946-44821c565031 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.131610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 481/938] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:08.136787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800266f9-bf18-4f28-803d-15138376f841 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.137080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800266f9-bf18-4f28-803d-15138376f841 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.137353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800266f9-bf18-4f28-803d-15138376f841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.155416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-800266f9-bf18-4f28-803d-15138376f841 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:08.156627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-800266f9-bf18-4f28-803d-15138376f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:08.227533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 458/939] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:08.249629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf80989-1905-40fc-b946-44821c565031 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:08.269560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:08.358779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf80989-1905-40fc-b946-44821c565031 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:08.394940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:08.395509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.395788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.396064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.396721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:08.398146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:08.407430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:08.409676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:08.411485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 482/940] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:08.421739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800266f9-bf18-4f28-803d-15138376f841 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:08.516372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:08.530429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800266f9-bf18-4f28-803d-15138376f841 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:08.565909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:08.566844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.567408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.567891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.568811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:08.569367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:08.570926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:08.576017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 459/941] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:08.582849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:08.667267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:08.691306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192a80-8ba5-4f59-badf-47f8b1a2a7c5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:08.731592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cd58b-cae8-41a8-af12-073efb51a4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:08.732227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cd58b-cae8-41a8-af12-073efb51a4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.732508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cd58b-cae8-41a8-af12-073efb51a4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.732818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cd58b-cae8-41a8-af12-073efb51a4b6 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.733496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cd58b-cae8-41a8-af12-073efb51a4b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:08.733798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cd58b-cae8-41a8-af12-073efb51a4b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:08.735285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cd58b-cae8-41a8-af12-073efb51a4b6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:08.741097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 483/942] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:08.746793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:08.851670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80643ca8-ab16-4822-a6b4-867ce0750b75 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:08.859919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dd8ce-894b-4936-8ba9-f161ef678147 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.860477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dd8ce-894b-4936-8ba9-f161ef678147 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.860993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dd8ce-894b-4936-8ba9-f161ef678147 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.862829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 484/943] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:08.869389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b2a7-a3c8-4574-88f2-6071bf431231 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.869703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b2a7-a3c8-4574-88f2-6071bf431231 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.869955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b2a7-a3c8-4574-88f2-6071bf431231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.892576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-61b9b2a7-a3c8-4574-88f2-6071bf431231 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:08.894549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-61b9b2a7-a3c8-4574-88f2-6071bf431231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:08.895848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:08.896874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:08.897399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:08.897935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:08.898883 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] RBAC: Adding query filter params (interface=public, service_id=d1158ca867234814865c8c93c3bfc4c6, region_id=RegionOne) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:08.899371 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:08.900979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:08.907310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 460/944] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:14:08 2022] GET /identity/v3/endpoints?service_id=d1158ca867234814865c8c93c3bfc4c6&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:08.912659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:08.933072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:08.973723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 485/945] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:09.000064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b2a7-a3c8-4574-88f2-6071bf431231 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:09.027650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b04ec0-8163-43c9-bbe7-71d0c706cd28 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:09.062160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:09.062779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.063061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.063340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.063980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:09.065895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:09.082330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 461/946] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:14:08 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:14:09.136706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b2a7-a3c8-4574-88f2-6071bf431231 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:09.157069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:09.181061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:09.182113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.182616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.183156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.184085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:09.185877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:09.189009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING keystone.server.flask.common [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 29 06:14:09.210067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.sql.core [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 06:14:09.210067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 06:14:09.210067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [parameters: {'id': '0b85fe3832174708a7f28db6514da9c9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 06:14:09.210067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80161) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b85fe3832174708a7f28db6514da9c9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b85fe3832174708a7f28db6514da9c9. Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 06:14:09.219077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 29 06:14:09.219943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application yield resource Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise self.value Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application session.commit() Jul 29 06:14:09.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.flush() Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application compat.raise_( Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 06:14:09.221868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 06:14:09.222768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 06:14:09.223589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [parameters: {'id': '0b85fe3832174708a7f28db6514da9c9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:09.224447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:09.225347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b85fe3832174708a7f28db6514da9c9. Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:09.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 486/947] 10.222.0.33 () {60 vars in 1140 bytes} [Fri Jul 29 06:14:08 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:09.232433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:09.323112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:09.343922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350612c-f0f7-460a-88fa-4b717aeba24d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:09.379975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cafe0b-c673-45fd-9652-df4d5251f39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:09.380735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cafe0b-c673-45fd-9652-df4d5251f39e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.381127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cafe0b-c673-45fd-9652-df4d5251f39e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.381421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cafe0b-c673-45fd-9652-df4d5251f39e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.388941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cafe0b-c673-45fd-9652-df4d5251f39e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:09.390390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cafe0b-c673-45fd-9652-df4d5251f39e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-53cafe0b-c673-45fd-9652-df4d5251f39e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:09.403220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:09.404158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:09.405072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 462/948] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:09.411240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:09.520640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd64f6-f887-4452-88b7-92b0f30c7ba2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:09.553112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:09.560021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d0b30-eed2-4fea-9dcc-2cab08bdffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:09.560683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0b30-eed2-4fea-9dcc-2cab08bdffbb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.560975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0b30-eed2-4fea-9dcc-2cab08bdffbb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.561256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0b30-eed2-4fea-9dcc-2cab08bdffbb None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.561981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0b30-eed2-4fea-9dcc-2cab08bdffbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:09.562286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0b30-eed2-4fea-9dcc-2cab08bdffbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:09.563913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0b30-eed2-4fea-9dcc-2cab08bdffbb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:09.588969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 487/949] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:14:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:09.811145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9e973-d689-40d3-b7cb-39d605a9fbdf None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.811757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9e973-d689-40d3-b7cb-39d605a9fbdf None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.811757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9e973-d689-40d3-b7cb-39d605a9fbdf None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.813330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 463/950] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:09.818894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5471ae-4d31-4dd2-ad3e-94f7daa3ec7b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.819191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5471ae-4d31-4dd2-ad3e-94f7daa3ec7b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.819478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5471ae-4d31-4dd2-ad3e-94f7daa3ec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.838290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6a5471ae-4d31-4dd2-ad3e-94f7daa3ec7b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:09.839805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6a5471ae-4d31-4dd2-ad3e-94f7daa3ec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:09.848679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c79a3-ce4a-456a-ae13-4729563dad40 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.849060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c79a3-ce4a-456a-ae13-4729563dad40 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.849347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c79a3-ce4a-456a-ae13-4729563dad40 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.851049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 464/951] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:09.856134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621d52b-1b9f-43a4-9fc1-f74eedcbcfa6 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:09.856418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621d52b-1b9f-43a4-9fc1-f74eedcbcfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:09.856735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621d52b-1b9f-43a4-9fc1-f74eedcbcfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:09.876292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5621d52b-1b9f-43a4-9fc1-f74eedcbcfa6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:09.877662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5621d52b-1b9f-43a4-9fc1-f74eedcbcfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:09.914488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 488/952] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:09.940458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5471ae-4d31-4dd2-ad3e-94f7daa3ec7b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:09.961435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 465/953] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:09.986768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621d52b-1b9f-43a4-9fc1-f74eedcbcfa6 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:10.066813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5471ae-4d31-4dd2-ad3e-94f7daa3ec7b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.089030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621d52b-1b9f-43a4-9fc1-f74eedcbcfa6 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.090660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:10.098063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:10.115212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.116161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.116935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.118283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.119033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:10.120710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:10.130300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:10.131541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:10.132828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:10.134733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 489/954] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:10.140479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:10.144790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.146570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.147032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.147464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.161925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:10.168410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:10.169619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:10.170675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:10.173736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 466/955] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 29 06:14:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:10.179451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:10.281545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10a201-e782-4122-9a0a-20236875a9d3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.307661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293237b3-2064-4763-bdeb-f8206c68fd47 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.320928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.321488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.321778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.322061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.322801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:10.323122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:10.324558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:10.335711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 490/956] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:14:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:10.337072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:10.349840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:10.358969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.360302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.360860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.361349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.362448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:10.363567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:10.365756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:10.382276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 467/957] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 29 06:14:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:10.383312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:10.406611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:10.466835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6c08e-e98f-41a5-85b4-8281c386efa8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.503929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.504504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.504830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.505107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.505803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:10.506121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:10.507649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:10.520641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 491/958] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:14:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:10.529454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c29907-8d7f-4884-9a40-ca430cb404b6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.569780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.570875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.571438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.571945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.585909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:10.587726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:10.604902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:10.605890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:10.606795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:10.606795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:10.606795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:10.606795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:10.613031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 468/959] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:10.616190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:10.742719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e793f-2964-4f3a-aa14-c83f35023681 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.757715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:10.778991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.779629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.779914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.780192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.781103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:10.781420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:10.782851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:10.797210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 492/960] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:10.803160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:10.854501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:10.914593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9424-8471-4277-93a4-7a127e87bff2 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:10.952389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:10.953095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:10.953384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:10.953667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:10.962334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:10.963780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:14:10.975552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:10.976527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:10.978750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 469/961] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:14:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:10.982802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.093004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436718b-21b4-43f3-948d-eea73fd0f381 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:11.130930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01541b9-7905-44f5-ba3e-b0c7f6c82540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:11.132569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01541b9-7905-44f5-ba3e-b0c7f6c82540 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.132569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01541b9-7905-44f5-ba3e-b0c7f6c82540 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.132569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01541b9-7905-44f5-ba3e-b0c7f6c82540 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.133048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01541b9-7905-44f5-ba3e-b0c7f6c82540 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:11.133209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01541b9-7905-44f5-ba3e-b0c7f6c82540 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:11.134547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01541b9-7905-44f5-ba3e-b0c7f6c82540 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:11.153107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 493/962] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 29 06:14:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:11.159325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.186294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f0ff5-dd0c-46e9-b456-b26bbbb33892 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.186620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f0ff5-dd0c-46e9-b456-b26bbbb33892 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.186872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f0ff5-dd0c-46e9-b456-b26bbbb33892 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.188400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 494/963] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:11.194227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af9e31-1d3d-423e-bf09-f884e9476a67 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.194755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af9e31-1d3d-423e-bf09-f884e9476a67 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.195232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af9e31-1d3d-423e-bf09-f884e9476a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.214673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-26af9e31-1d3d-423e-bf09-f884e9476a67 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:11.216198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-26af9e31-1d3d-423e-bf09-f884e9476a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:11.244041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:11.285077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2eb1ee-ad21-4a2b-ae18-a38869b95cbe None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:11.294681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:11.304876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 495/964] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:11.323838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af9e31-1d3d-423e-bf09-f884e9476a67 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.326610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:11.327253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.327542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.327825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.404683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:11.406209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:11.432939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af9e31-1d3d-423e-bf09-f884e9476a67 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:11.473167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:11.474636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.475153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.475625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.476455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:11.478164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:11.488050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:11.489135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:11.489971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:11.497449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 496/965] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:11.498534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.519752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 470/966] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:11 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:11.520893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.528945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:11.530160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:11.608035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f22b6-05c9-4928-a4cd-247b5e379e89 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:11.634733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f24a6-8265-496f-9d6e-ca911e2c4af5 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:11.647228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:11.648100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.648526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.649002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.649903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:11.650408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:11.652075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:11.657347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 497/967] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:11.663148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.676863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d97300-440d-4f9f-9e0c-eb880fd43447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:11.678299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97300-440d-4f9f-9e0c-eb880fd43447 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.679039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97300-440d-4f9f-9e0c-eb880fd43447 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.679492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97300-440d-4f9f-9e0c-eb880fd43447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.695235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d97300-440d-4f9f-9e0c-eb880fd43447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:11.697537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d97300-440d-4f9f-9e0c-eb880fd43447 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-b3d97300-440d-4f9f-9e0c-eb880fd43447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:11.718456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:11.719408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:11.720478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:11.720478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:11.720478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:11.720478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:11.720478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 471/968] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:11.724429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f98e01-521e-42d2-b044-7f422245df05 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.724833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f98e01-521e-42d2-b044-7f422245df05 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.725383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f98e01-521e-42d2-b044-7f422245df05 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.726633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 472/969] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:11.727980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f98e01-521e-42d2-b044-7f422245df05 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.778514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:11.790493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7b3d0-3ae3-4fa3-84aa-fa2a77a88606 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:11.792302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:11.827313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca22cf-4b79-424c-8372-c24e1717ebd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:11.828038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca22cf-4b79-424c-8372-c24e1717ebd7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.828233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca22cf-4b79-424c-8372-c24e1717ebd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.828494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca22cf-4b79-424c-8372-c24e1717ebd7 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.829196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca22cf-4b79-424c-8372-c24e1717ebd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:11.829508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca22cf-4b79-424c-8372-c24e1717ebd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:11.831799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca22cf-4b79-424c-8372-c24e1717ebd7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:11.841422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 498/970] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:11.841422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ab3b5-b79e-4f13-bd6a-f2ce3f5c3226 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.841422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ab3b5-b79e-4f13-bd6a-f2ce3f5c3226 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.841422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ab3b5-b79e-4f13-bd6a-f2ce3f5c3226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.846828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f98e01-521e-42d2-b044-7f422245df05 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:11.856940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-969ab3b5-b79e-4f13-bd6a-f2ce3f5c3226 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:11.858129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-969ab3b5-b79e-4f13-bd6a-f2ce3f5c3226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:11.882455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:11.883073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:11.883356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:11.883630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:11.884314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:11.884649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:11.885996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:11.902396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 473/971] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:11.902396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:11.936678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 499/972] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:11.936678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ab3b5-b79e-4f13-bd6a-f2ce3f5c3226 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:12.003711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71815568-db3f-4f65-9065-b6da57a1a2b4 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.020820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:12.031151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:12.039458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.040084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.040365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.040664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.041361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.042719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.050209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ab3b5-b79e-4f13-bd6a-f2ce3f5c3226 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.072590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 474/973] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:14:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:12.072590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:12.116875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.117539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.117820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.118091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.131816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.134587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:12.153274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:12.154253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:12.155329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 500/974] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:12.161325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:12.200627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf86a9d-1f1f-447b-a9a7-afd0ea3e8555 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.241617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.242610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.242896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.243161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.243842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.245363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.247904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING keystone.server.flask.common [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 29 06:14:12.261356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.sql.core [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 06:14:12.261356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 06:14:12.261356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [parameters: {'user_id': 'ede053970d584af68f739ddc41ca8aaf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 06:14:12.261356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80162) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 06:14:12.278811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab6651-361f-4b17-965c-78cf258487d8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application conn.query(q) Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application result.read() Jul 29 06:14:12.279133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 06:14:12.280033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application next(self.gen) Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application yield resource Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application next(self.gen) Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise self.value Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application session.commit() Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 06:14:12.282331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self.session.flush() Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application compat.raise_( Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 06:14:12.283288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 06:14:12.287497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application conn.query(q) Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application result.read() Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 06:14:12.289766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ede053970d584af68f739ddc41ca8aaf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:12.291978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 29 06:14:12.294711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 475/975] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 29 06:14:12 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:12.297290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:12.328749 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.329634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.329920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.332219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:12.332357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.334055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:12.334425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.336195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.350299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:12.356734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 501/976] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:12.362741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:12.420671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff9f9-3179-4d04-a857-f1183f9fdea5 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.482018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.482710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.483115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.483721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.495116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.497795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.500183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf4c6a-cd57-455a-a8d2-eab7b01037ba None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:14:12.514277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:12.515305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 06:14:12.516229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:12.520652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 476/977] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:14:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:12.521737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:12.551889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26e06e-726d-4793-a217-e5822998a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.553224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26e06e-726d-4793-a217-e5822998a56f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.553916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26e06e-726d-4793-a217-e5822998a56f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.554377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26e06e-726d-4793-a217-e5822998a56f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.555553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26e06e-726d-4793-a217-e5822998a56f None admin] RBAC: Adding query filter params (role.id=ab91e6fc0e6942c2bd23a93741d5b1d8, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=a4141b562f1442efaa032122666724df) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:12.556310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26e06e-726d-4793-a217-e5822998a56f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.558180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26e06e-726d-4793-a217-e5822998a56f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.566511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 502/978] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:12 2022] GET /identity/v3/role_assignments?user.id=a4141b562f1442efaa032122666724df&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=ab91e6fc0e6942c2bd23a93741d5b1d8 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:12.609241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06f231-4c03-4d0f-8d1d-3b7f6763ba44 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.609798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06f231-4c03-4d0f-8d1d-3b7f6763ba44 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.610265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06f231-4c03-4d0f-8d1d-3b7f6763ba44 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.617942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 503/979] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:12.619993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944ed94-c71f-417b-b727-eb43d06d13f1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.620279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944ed94-c71f-417b-b727-eb43d06d13f1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.620590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944ed94-c71f-417b-b727-eb43d06d13f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.646712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d944ed94-c71f-417b-b727-eb43d06d13f1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:12.648218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d944ed94-c71f-417b-b727-eb43d06d13f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:12.648971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:12.656631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ca340-bfb3-48ec-820a-7430111ef62b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.702018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.702673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.702958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.703238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.704024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:12.704332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.706349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.742716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:12.751503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 477/980] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 29 06:14:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:12.756648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 504/981] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:12.775053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:12.897766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0462943-fc1a-43dd-89aa-247e254546fe None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:12.936883 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8fcfe-39f4-4e2e-bc4d-c88318422740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:12.937516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8fcfe-39f4-4e2e-bc4d-c88318422740 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:12.937805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8fcfe-39f4-4e2e-bc4d-c88318422740 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:12.938085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8fcfe-39f4-4e2e-bc4d-c88318422740 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:12.938795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8fcfe-39f4-4e2e-bc4d-c88318422740 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:12.940839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8fcfe-39f4-4e2e-bc4d-c88318422740 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:12.957617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 478/982] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:14:12 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:14.214057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b30a9-edcd-4f81-ad36-e1f980becedc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.214620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b30a9-edcd-4f81-ad36-e1f980becedc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.214620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b30a9-edcd-4f81-ad36-e1f980becedc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.216148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 505/983] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:14.221774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb53635-7eaa-4c0b-83f7-a73b1f6e8299 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.222195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb53635-7eaa-4c0b-83f7-a73b1f6e8299 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.222390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb53635-7eaa-4c0b-83f7-a73b1f6e8299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.240908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-bbb53635-7eaa-4c0b-83f7-a73b1f6e8299 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:14.242419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bbb53635-7eaa-4c0b-83f7-a73b1f6e8299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:14.252529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:14.319949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 479/984] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:14.341993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b30a9-edcd-4f81-ad36-e1f980becedc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:14.447211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b30a9-edcd-4f81-ad36-e1f980becedc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:14.483008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:14.483575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.483861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.484140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.484802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:14.486175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:14.495440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:14.496427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:14.497292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:14.500877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 506/985] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:14.502659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb53635-7eaa-4c0b-83f7-a73b1f6e8299 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:14.602814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb53635-7eaa-4c0b-83f7-a73b1f6e8299 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:14.637725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdb0fd-a11c-4885-b811-b3a635fcdc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:14.638357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdb0fd-a11c-4885-b811-b3a635fcdc6d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.638641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdb0fd-a11c-4885-b811-b3a635fcdc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.638985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdb0fd-a11c-4885-b811-b3a635fcdc6d None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.639686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdb0fd-a11c-4885-b811-b3a635fcdc6d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:14.639994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdb0fd-a11c-4885-b811-b3a635fcdc6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:14.641402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdb0fd-a11c-4885-b811-b3a635fcdc6d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:14.647187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 480/986] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:14.652650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:14.699641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e9f8f-b90a-4487-a8ce-cc065c3f949a None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.699968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e9f8f-b90a-4487-a8ce-cc065c3f949a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.700205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e9f8f-b90a-4487-a8ce-cc065c3f949a None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.701657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 481/987] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:14.707021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce8f65-7644-45fe-b631-5ed74b642caf None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.707304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce8f65-7644-45fe-b631-5ed74b642caf None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.707580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce8f65-7644-45fe-b631-5ed74b642caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.713844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:14.715831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:14.732245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f4ce8f65-7644-45fe-b631-5ed74b642caf None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:14.733892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f4ce8f65-7644-45fe-b631-5ed74b642caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:14.763456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce818c-681e-4224-8f56-b055321e3f85 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:14.799492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:14.800123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.800407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.800722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.801541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:14.801846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:14.803213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:14.808120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 507/988] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:14.813845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:14.814276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 482/989] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:14.837624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce8f65-7644-45fe-b631-5ed74b642caf None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:14.922026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12492021-13a3-41c7-8ee1-7ae075f151af None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:14.947147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:14.948357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce8f65-7644-45fe-b631-5ed74b642caf None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:14.962463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:14.963094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.963378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.963657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.964423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] RBAC: Adding query filter params (interface=admin, service_id=d1158ca867234814865c8c93c3bfc4c6, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:14.964809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:14.966187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:14.975649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 508/990] 10.222.0.33 () {56 vars in 1249 bytes} [Fri Jul 29 06:14:14 2022] GET /identity/v3/endpoints?service_id=d1158ca867234814865c8c93c3bfc4c6&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:14.977040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:14.988296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec117d-baa4-4306-a626-960e4d54c79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:14.988937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec117d-baa4-4306-a626-960e4d54c79a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:14.989219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec117d-baa4-4306-a626-960e4d54c79a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:14.989811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec117d-baa4-4306-a626-960e4d54c79a None admin] PATH_INFO: `/v3/services/image` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:14.990516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec117d-baa4-4306-a626-960e4d54c79a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:14.991789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec117d-baa4-4306-a626-960e4d54c79a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:14.993625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-1fec117d-baa4-4306-a626-960e4d54c79a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:15.002045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:15.003621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:15.004449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 483/991] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jul 29 06:14:14 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:15.005283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001301a-6640-4dbd-80ff-575e7024ddcb None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.005283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001301a-6640-4dbd-80ff-575e7024ddcb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.005422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001301a-6640-4dbd-80ff-575e7024ddcb None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.006629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 484/992] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:15.009529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001301a-6640-4dbd-80ff-575e7024ddcb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:15.082534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae77170-e8a3-4eff-800a-1214f8d13120 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:15.113611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001301a-6640-4dbd-80ff-575e7024ddcb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:15.116777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0e8c0-becd-4d16-9fd0-572dcb315dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:15.117410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0e8c0-becd-4d16-9fd0-572dcb315dfe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.117699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0e8c0-becd-4d16-9fd0-572dcb315dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.117981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0e8c0-becd-4d16-9fd0-572dcb315dfe None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.118634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0e8c0-becd-4d16-9fd0-572dcb315dfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:15.120007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0e8c0-becd-4d16-9fd0-572dcb315dfe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:15.128143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 509/993] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:14:14 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:14:15.129339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e11b8-10dc-4259-b91c-d392e84fc9f1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.129629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e11b8-10dc-4259-b91c-d392e84fc9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.129908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e11b8-10dc-4259-b91c-d392e84fc9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.149055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-848e11b8-10dc-4259-b91c-d392e84fc9f1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:15.150200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:15.150337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-848e11b8-10dc-4259-b91c-d392e84fc9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:15.150845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.151125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.151399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.152087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] RBAC: Adding query filter params (name=image) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:15.152411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:15.155146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:15.166070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 485/994] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:15.167424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:15.206715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:15.229874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 510/995] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:15.241612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:15.253123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e11b8-10dc-4259-b91c-d392e84fc9f1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:15.286512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdf91e-e789-4241-b4cb-a7bf8e560cdf None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:15.326841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:15.327471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.327753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.328026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.328724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:15.329031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:15.331100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:15.335253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 486/996] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:15.369147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e11b8-10dc-4259-b91c-d392e84fc9f1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:15.407403 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:15.408066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.408347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.408816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.409456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:15.410860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:15.420928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:15.422226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:15.423081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 511/997] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:15.429091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:15.542057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8212fe-00b6-43c2-86f6-3d6a3398c9a6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:15.568596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:15.578104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:15.578728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.579011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.579285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.579960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:15.580265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:15.582300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:15.591938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 487/998] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:14:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:15.593489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:15.642288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:15.700835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c662d8-746d-4bdd-9b3d-757221452796 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:15.744112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:15.744770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.745058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.745337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.746018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:15.746332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:15.747704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:15.757728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 512/999] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:14:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:15.759566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:15.864151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718f6af-8e6d-4691-b49c-9a628a53a493 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:15.897984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c938012-0e10-49cf-af98-f6520ba83726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:15.898613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938012-0e10-49cf-af98-f6520ba83726 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:15.898895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938012-0e10-49cf-af98-f6520ba83726 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:15.899170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938012-0e10-49cf-af98-f6520ba83726 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:15.899921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938012-0e10-49cf-af98-f6520ba83726 None admin] RBAC: Adding query filter params (interface=public, service_id=01d887b7661147079184ac21784a1873, region_id=RegionOne) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:15.900228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938012-0e10-49cf-af98-f6520ba83726 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:15.901623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938012-0e10-49cf-af98-f6520ba83726 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:15.911314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 488/1000] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:14:15 2022] GET /identity/v3/endpoints?service_id=01d887b7661147079184ac21784a1873&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:15.913332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:16.025711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df616e-1ade-4975-a439-5fe3b9aa3ee6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:16.035741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:16.061551 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e6654-aee9-4952-ab3d-95e1475ae319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:16.062172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e6654-aee9-4952-ab3d-95e1475ae319 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:16.062448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e6654-aee9-4952-ab3d-95e1475ae319 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:16.062718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e6654-aee9-4952-ab3d-95e1475ae319 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:16.063358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e6654-aee9-4952-ab3d-95e1475ae319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:16.064745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e6654-aee9-4952-ab3d-95e1475ae319 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:16.076772 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 513/1001] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:14:15 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:14:17.143642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ef6e1-e8e1-4b44-a33d-b5ce4cf976a6 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.144190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ef6e1-e8e1-4b44-a33d-b5ce4cf976a6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.144190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ef6e1-e8e1-4b44-a33d-b5ce4cf976a6 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.145601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 489/1002] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:17.150677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443f74a-3a97-4a7d-9cf8-1c1c7e1b5e07 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.151072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443f74a-3a97-4a7d-9cf8-1c1c7e1b5e07 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.151253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443f74a-3a97-4a7d-9cf8-1c1c7e1b5e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.169640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b443f74a-3a97-4a7d-9cf8-1c1c7e1b5e07 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:17.170806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b443f74a-3a97-4a7d-9cf8-1c1c7e1b5e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:17.241927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 514/1003] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:17.264006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ef6e1-e8e1-4b44-a33d-b5ce4cf976a6 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:17.321668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:17.368281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ef6e1-e8e1-4b44-a33d-b5ce4cf976a6 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:17.374768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc82d0-f330-43e0-93e6-494b895cd003 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.375062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc82d0-f330-43e0-93e6-494b895cd003 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.375342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc82d0-f330-43e0-93e6-494b895cd003 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.377053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 515/1004] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:17.382314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73748229-0c9c-46d1-982e-bac9338e1694 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.382605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73748229-0c9c-46d1-982e-bac9338e1694 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.382886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73748229-0c9c-46d1-982e-bac9338e1694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.400911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-73748229-0c9c-46d1-982e-bac9338e1694 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:17.402175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-73748229-0c9c-46d1-982e-bac9338e1694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:17.405190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:17.405830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.406113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.406392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.406994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:17.408339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:17.417295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:17.418293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:17.419145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 490/1005] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 29 06:14:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:17.424951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:17.492560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 516/1006] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:17.503169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73748229-0c9c-46d1-982e-bac9338e1694 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:17.535812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:17.537210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf33747-d749-44b9-88bb-19676206f89c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:17.556891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:17.574981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:17.575613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.575891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.576167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.576935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:17.577493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:17.579281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:17.584805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 491/1007] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:17.590294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:17.612101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73748229-0c9c-46d1-982e-bac9338e1694 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:17.651003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:17.651556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.651845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.652128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.652852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:17.654233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:17.672759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 517/1008] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:14:17 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:14:17.701152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bd449-ccf1-46ab-883d-0e6dbb1c381b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:17.756524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e9419-9ce6-480e-a44d-5974cb4f98a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:17.757227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e9419-9ce6-480e-a44d-5974cb4f98a6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.757516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e9419-9ce6-480e-a44d-5974cb4f98a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.757790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e9419-9ce6-480e-a44d-5974cb4f98a6 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.758477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e9419-9ce6-480e-a44d-5974cb4f98a6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:17.758778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e9419-9ce6-480e-a44d-5974cb4f98a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:17.760161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e9419-9ce6-480e-a44d-5974cb4f98a6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:17.771068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 492/1009] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 29 06:14:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:17.773061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:17.874838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:17.885398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8edce-7f29-4c3f-acd2-7f2bdec3462a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:17.920635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27be6a-180b-41d1-a211-62bd7ac25bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:17.921188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27be6a-180b-41d1-a211-62bd7ac25bec None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:17.921467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27be6a-180b-41d1-a211-62bd7ac25bec None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:17.921742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27be6a-180b-41d1-a211-62bd7ac25bec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:17.922471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27be6a-180b-41d1-a211-62bd7ac25bec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:17.923830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27be6a-180b-41d1-a211-62bd7ac25bec None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:17.948561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 518/1010] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:14:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:18.116268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabc283-1bf1-4337-a379-98bea4b2e3d7 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:18.116639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabc283-1bf1-4337-a379-98bea4b2e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:18.116888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabc283-1bf1-4337-a379-98bea4b2e3d7 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:18.118385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 493/1011] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:18.123381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d0b6-aba1-4dad-9e90-70dbc607366d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:18.123683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d0b6-aba1-4dad-9e90-70dbc607366d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:18.123969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d0b6-aba1-4dad-9e90-70dbc607366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:18.142739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-78a6d0b6-aba1-4dad-9e90-70dbc607366d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:18.144567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-78a6d0b6-aba1-4dad-9e90-70dbc607366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:18.220320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 519/1012] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:18 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:18.242969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabc283-1bf1-4337-a379-98bea4b2e3d7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:18.290983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:18.354293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabc283-1bf1-4337-a379-98bea4b2e3d7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:18.388963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:18.389614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:18.389898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:18.390176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:18.390804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:18.392170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:18.402228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:18.403299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:18.404152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 494/1013] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:18.411724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d0b6-aba1-4dad-9e90-70dbc607366d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:18.490171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:18.515106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d0b6-aba1-4dad-9e90-70dbc607366d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:18.551397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:18.552716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:18.552716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:18.552716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:18.553174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:18.553584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:18.555065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:18.565179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 520/1014] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:14:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:18.566856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:18.671927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3628-0141-4e9d-bc01-3f5a5289260e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:18.679232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:18.708872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68761e-7224-42bc-9b38-98910c6e0cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:18.709502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68761e-7224-42bc-9b38-98910c6e0cde None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:18.709785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68761e-7224-42bc-9b38-98910c6e0cde None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:18.710061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68761e-7224-42bc-9b38-98910c6e0cde None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:18.710757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68761e-7224-42bc-9b38-98910c6e0cde None admin] RBAC: Adding query filter params (type=compute) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:18.711118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68761e-7224-42bc-9b38-98910c6e0cde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:18.712462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68761e-7224-42bc-9b38-98910c6e0cde None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:18.723027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 495/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 29 06:14:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:18.724369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:18.832869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761763ba-b22b-47fb-9fdd-f39bc5a2794d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:18.867849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f9122-1647-4641-945a-29778ea4719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:18.868469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f9122-1647-4641-945a-29778ea4719c None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:18.868784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f9122-1647-4641-945a-29778ea4719c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:18.869062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f9122-1647-4641-945a-29778ea4719c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:18.869766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f9122-1647-4641-945a-29778ea4719c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:18.871128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f9122-1647-4641-945a-29778ea4719c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:18.886089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:18.894032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 521/1016] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:14:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:19.729878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1d7ea-07b2-4645-ae48-a7788f64f27d None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:19.729878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1d7ea-07b2-4645-ae48-a7788f64f27d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:19.729878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1d7ea-07b2-4645-ae48-a7788f64f27d None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:19.730612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 496/1017] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:19.735398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23679d1-86c4-4cff-9daf-b54e8161d4df None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:19.735796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23679d1-86c4-4cff-9daf-b54e8161d4df None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:19.735956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23679d1-86c4-4cff-9daf-b54e8161d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:19.754334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c23679d1-86c4-4cff-9daf-b54e8161d4df None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:19.755521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c23679d1-86c4-4cff-9daf-b54e8161d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:19.832059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 522/1018] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:19.854953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1d7ea-07b2-4645-ae48-a7788f64f27d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:19.960859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1d7ea-07b2-4645-ae48-a7788f64f27d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:19.969753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba1442-b540-4933-a4cf-408efaa6e47f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:19.970048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba1442-b540-4933-a4cf-408efaa6e47f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:19.970330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba1442-b540-4933-a4cf-408efaa6e47f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:19.972996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 523/1019] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:19.978640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8fc87-2f6b-4981-9689-903505e2b3f8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:19.979076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8fc87-2f6b-4981-9689-903505e2b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:19.979361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8fc87-2f6b-4981-9689-903505e2b3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:19.997347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c5c8fc87-2f6b-4981-9689-903505e2b3f8 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:19.998452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c5c8fc87-2f6b-4981-9689-903505e2b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:19.999278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:19.999982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:20.000266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:20.000579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] PATH_INFO: `/v3/services/image` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:20.001195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:20.003061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:20.011965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:20.013186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:20.014030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 497/1020] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jul 29 06:14:19 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:20.019924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:20.054953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:20.076599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 524/1021] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:20.099270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8fc87-2f6b-4981-9689-903505e2b3f8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:20.128149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b13eb8-d582-4697-bf18-c8d365952c75 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:20.159119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:20.165097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:20.165729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:20.166014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:20.166293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:20.166992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] RBAC: Adding query filter params (name=image) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:20.167283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:20.168667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:20.178598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 498/1022] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:20.179247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:20.206158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8fc87-2f6b-4981-9689-903505e2b3f8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:20.242787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:20.243423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:20.243694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:20.243969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:20.245131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:20.246551 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:20.248390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 525/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:20.254636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:20.284328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:20.289280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdaebd-c5ab-4a75-b696-4b75e6a3044b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:20.328898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:20.329522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:20.329807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:20.330083 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:20.331623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] RBAC: Adding query filter params (type=image) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:20.332036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:20.333886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:20.340988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 499/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:20.346841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:20.359903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0dae-8bb7-4a50-a6da-d25132279cf3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:20.384783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:20.398487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:20.399107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:20.399393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:20.399669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:20.400349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:20.401814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:20.432842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 526/1025] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:14:20 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:14:20.470767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10f9f6-1841-484b-94ed-0e6b20a52e71 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:20.508526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8e56a-95ca-4f67-8ac8-6abcf8060d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:20.509183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e56a-95ca-4f67-8ac8-6abcf8060d46 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:20.509467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e56a-95ca-4f67-8ac8-6abcf8060d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:20.509738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e56a-95ca-4f67-8ac8-6abcf8060d46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:20.510503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e56a-95ca-4f67-8ac8-6abcf8060d46 None admin] RBAC: Adding query filter params (interface=public, service_id=b1de706a0add4cb4851402be2bee10b1, region_id=RegionOne) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:20.510813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e56a-95ca-4f67-8ac8-6abcf8060d46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:20.512190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e56a-95ca-4f67-8ac8-6abcf8060d46 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:20.523202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 500/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:14:20 2022] GET /identity/v3/endpoints?service_id=b1de706a0add4cb4851402be2bee10b1&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:20.524806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:20.647583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3eceb-4845-45c4-a514-1f2a297afeda None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:20.684778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a20bb9-8665-466b-951b-3af51791d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:20.685470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a20bb9-8665-466b-951b-3af51791d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:20.685750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a20bb9-8665-466b-951b-3af51791d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:20.686022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a20bb9-8665-466b-951b-3af51791d9f9 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:20.686720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a20bb9-8665-466b-951b-3af51791d9f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:20.688090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a20bb9-8665-466b-951b-3af51791d9f9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:20.695920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 527/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:14:20 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:14:21.006973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ab46-5c49-46cb-980a-d012fe22ffc1 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:21.007520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ab46-5c49-46cb-980a-d012fe22ffc1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:21.007520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ab46-5c49-46cb-980a-d012fe22ffc1 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:21.008953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 501/1028] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:21.014110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de679cbb-2963-4989-ad56-98d6f434f996 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:21.014505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de679cbb-2963-4989-ad56-98d6f434f996 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:21.014672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de679cbb-2963-4989-ad56-98d6f434f996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:21.033271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-de679cbb-2963-4989-ad56-98d6f434f996 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:21.035376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-de679cbb-2963-4989-ad56-98d6f434f996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:21.067116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:21.107644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 528/1029] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:21.131522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3ab46-5c49-46cb-980a-d012fe22ffc1 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:21.162994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:21.243879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3ab46-5c49-46cb-980a-d012fe22ffc1 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:21.278279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:21.278847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:21.279128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:21.279400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:21.290333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:21.295316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:21.296219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:21.297107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:21.298777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 502/1030] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 29 06:14:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:21.303745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de679cbb-2963-4989-ad56-98d6f434f996 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:21.405199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de679cbb-2963-4989-ad56-98d6f434f996 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:21.440961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:21.441710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:21.442004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:21.442284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:21.442994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:21.443301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:21.444761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:21.459078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 529/1031] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 29 06:14:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:21.460393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:21.550150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:21.565208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcbb18-ed8d-404e-b7cf-703566d356ea None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:21.600622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:21.601273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:21.601554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:21.601823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:21.612113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:21.613554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:21.628125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:21.629073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:21.634584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:21.634584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:21.634584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:21.634584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:21.634584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 503/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:21.636393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:21.669281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:21.748823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab96f-1463-46cb-bddb-41a19fd58ac4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:21.784314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:21.784976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:21.785264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:21.785544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:21.786232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:21.786541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:21.787910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:21.800494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 530/1033] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:21.806497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:21.909701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78893-1e2b-473e-9785-b2865d3f6b06 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:21.943170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:21.943819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:21.944107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:21.944386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:21.951862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:21.953325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:21.955025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:14:21.967038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:21.968033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:21.974556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 504/1034] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:14:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:21.975178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:22.066406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:22.077334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f7f5-8831-4435-9e2e-3ce0d1aadc1e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:22.113214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:22.113837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.114116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.114390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.115165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:22.115467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:22.116917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:22.138686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 531/1035] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:14:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:22.139967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:22.242280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822fcd5-088b-49c9-a4fd-891e3f633cab None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:22.276006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229cecd-5884-45c0-9b73-f5144a31c0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:22.276656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229cecd-5884-45c0-9b73-f5144a31c0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.276940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229cecd-5884-45c0-9b73-f5144a31c0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.277215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229cecd-5884-45c0-9b73-f5144a31c0e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.344093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:22.349456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229cecd-5884-45c0-9b73-f5144a31c0e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:22.350903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229cecd-5884-45c0-9b73-f5144a31c0e1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:22.462393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 505/1036] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:22 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 06:14:22.463704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:22.503564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b9e7c-e2de-463c-a278-ee8ef3d0539e None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.503907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b9e7c-e2de-463c-a278-ee8ef3d0539e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.504141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b9e7c-e2de-463c-a278-ee8ef3d0539e None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.505659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 506/1037] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:22.510683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a2f3e-84e7-42bf-bc0f-c5129b061b0c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.510968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a2f3e-84e7-42bf-bc0f-c5129b061b0c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.511244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a2f3e-84e7-42bf-bc0f-c5129b061b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.529334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a03a2f3e-84e7-42bf-bc0f-c5129b061b0c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:22.530500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a03a2f3e-84e7-42bf-bc0f-c5129b061b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:22.587376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1906-2bef-4fd1-9175-fa302f98967d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:22.606842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 507/1038] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:22.627281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:22.628085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.628370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.628677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.631803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a2f3e-84e7-42bf-bc0f-c5129b061b0c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:22.639366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:22.641294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:22.651990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:22.660224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:22.661315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:22.662207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:22.662207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:22.662207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:22.662207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:22.664020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:22.666304 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 532/1039] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:22.667183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:22.747807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a2f3e-84e7-42bf-bc0f-c5129b061b0c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:22.775633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321c1fc-1c8f-48e8-8125-273de5d30f24 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:22.781221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc5e14-6c0b-4bf5-aa65-cac4951bd5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:22.781864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5e14-6c0b-4bf5-aa65-cac4951bd5aa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.782149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5e14-6c0b-4bf5-aa65-cac4951bd5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.782483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5e14-6c0b-4bf5-aa65-cac4951bd5aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.784217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc5e14-6c0b-4bf5-aa65-cac4951bd5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:22.785674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc5e14-6c0b-4bf5-aa65-cac4951bd5aa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:22.787293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 508/1040] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:22.788389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2de4ab-0379-45cb-925e-e3414bc7f507 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.788699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2de4ab-0379-45cb-925e-e3414bc7f507 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.788978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2de4ab-0379-45cb-925e-e3414bc7f507 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.790359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 509/1041] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:22.792831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2de4ab-0379-45cb-925e-e3414bc7f507 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:22.812479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f03d31-f1cd-40b2-b25b-698b0345087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:22.813139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f03d31-f1cd-40b2-b25b-698b0345087a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.813428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f03d31-f1cd-40b2-b25b-698b0345087a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.813710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f03d31-f1cd-40b2-b25b-698b0345087a None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.814402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f03d31-f1cd-40b2-b25b-698b0345087a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:22.814714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f03d31-f1cd-40b2-b25b-698b0345087a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:22.816274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f03d31-f1cd-40b2-b25b-698b0345087a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:22.828101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 533/1042] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:22.829238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8344a-61e2-4243-9a8d-01d31982e4aa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.829537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8344a-61e2-4243-9a8d-01d31982e4aa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.829820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8344a-61e2-4243-9a8d-01d31982e4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.849115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c0e8344a-61e2-4243-9a8d-01d31982e4aa None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:22.850295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c0e8344a-61e2-4243-9a8d-01d31982e4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:22.886758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:22.894675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2de4ab-0379-45cb-925e-e3414bc7f507 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:22.906235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:22.917979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 534/1043] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:22.918785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8344a-61e2-4243-9a8d-01d31982e4aa None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:22.936460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:22.937152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:22.937404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:22.937680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:22.938363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:22.940172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:22.962565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 510/1044] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:14:22 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:14:22.963489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:23.062295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8344a-61e2-4243-9a8d-01d31982e4aa None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:23.085629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfb76d-1919-4367-8c9e-5aa52650733e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:23.118809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:23.119748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:23.120750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:23.121216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:23.130314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:23.131941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:23.140023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:23.141361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:23.141875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:23.142373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:23.144049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:23.145184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:23.146071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:23.146980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 535/1045] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:23.148599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:23.157489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:23.161438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:23.162409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 06:14:23.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:23.166104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 511/1046] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jul 29 06:14:22 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:23.174810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:23.204130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:23.228955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:23.284632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee2533-39ec-44f9-9fd6-11e720138bde None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:23.308915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a8c8-d8ff-48d4-bf89-cef96730a5d2 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:23.329510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:23.330994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:23.331287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:23.331545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:23.332564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:23.332676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:23.334098 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:23.350291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 536/1047] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:14:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:23.355583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:23.355886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:23.356221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:23.356508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:23.356815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:23.357518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:23.357808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:23.359168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:23.369761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 512/1048] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:23.375365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:23.481268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:23.498703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d21ba-e4be-430c-b3e5-2fe83fb3e648 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:23.505437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf100d0-48fb-4d2a-b4c4-d1e740c71ae4 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:23.510428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:23.539466 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:23.540097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:23.540383 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:23.540695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:23.541483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] RBAC: Adding query filter params (role.id=ab91e6fc0e6942c2bd23a93741d5b1d8, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bbe64416672b408f8cdf93220cc31ef5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:23.541715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4791e-b851-48f5-ad37-f652b98c8700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:23.541828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:23.542345 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4791e-b851-48f5-ad37-f652b98c8700 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:23.544406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4791e-b851-48f5-ad37-f652b98c8700 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:23.544406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4791e-b851-48f5-ad37-f652b98c8700 None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:23.544611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:23.544698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4791e-b851-48f5-ad37-f652b98c8700 None admin] RBAC: Adding query filter params (type=image) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:23.544698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4791e-b851-48f5-ad37-f652b98c8700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:23.547464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4791e-b851-48f5-ad37-f652b98c8700 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:23.551217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 537/1049] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:23 2022] GET /identity/v3/role_assignments?user.id=bbe64416672b408f8cdf93220cc31ef5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=ab91e6fc0e6942c2bd23a93741d5b1d8 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:23.551578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 513/1050] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:23.560887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:23.674488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec29be7-f76e-433d-8e3f-3db355d32df8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:23.711845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efc74b-fa0b-4e2b-8153-f92da9737205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:23.712473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efc74b-fa0b-4e2b-8153-f92da9737205 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:23.712802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efc74b-fa0b-4e2b-8153-f92da9737205 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:23.713081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efc74b-fa0b-4e2b-8153-f92da9737205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:23.713790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efc74b-fa0b-4e2b-8153-f92da9737205 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:23.715145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efc74b-fa0b-4e2b-8153-f92da9737205 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:23.744475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 538/1051] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:14:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:14:25.064098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd3ec-5eb9-41c5-a1cf-39c1786fd73a None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.064790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd3ec-5eb9-41c5-a1cf-39c1786fd73a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.064790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd3ec-5eb9-41c5-a1cf-39c1786fd73a None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.066101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 514/1052] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:25.071388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f053db0-e2f4-413a-951d-8e280839abcd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.071773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f053db0-e2f4-413a-951d-8e280839abcd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.072016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f053db0-e2f4-413a-951d-8e280839abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.091283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4f053db0-e2f4-413a-951d-8e280839abcd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:25.092471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4f053db0-e2f4-413a-951d-8e280839abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:25.117734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:25.170792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 539/1053] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:25.194306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fd3ec-5eb9-41c5-a1cf-39c1786fd73a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:25.298183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fd3ec-5eb9-41c5-a1cf-39c1786fd73a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:25.333053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:25.333717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.334002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.334281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.344529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:25.350146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:25.351118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:25.351948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:25.353211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 515/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:25.357520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f053db0-e2f4-413a-951d-8e280839abcd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:25.460027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f053db0-e2f4-413a-951d-8e280839abcd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:25.495370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19b0ad-bf0c-4287-b92f-fb362bd2c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:25.495998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19b0ad-bf0c-4287-b92f-fb362bd2c74f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.496281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19b0ad-bf0c-4287-b92f-fb362bd2c74f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.496642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19b0ad-bf0c-4287-b92f-fb362bd2c74f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.497357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19b0ad-bf0c-4287-b92f-fb362bd2c74f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:25.497665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19b0ad-bf0c-4287-b92f-fb362bd2c74f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:25.499034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19b0ad-bf0c-4287-b92f-fb362bd2c74f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:25.508709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 540/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:25.513694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:25.518779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:25.572223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd8c11-e48f-4668-b3d9-a167e91394d8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.572601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd8c11-e48f-4668-b3d9-a167e91394d8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.572810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd8c11-e48f-4668-b3d9-a167e91394d8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.576569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 541/1056] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:25.578970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060e540-71ab-4367-95d2-8f95a6e769db None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.579373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060e540-71ab-4367-95d2-8f95a6e769db None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.579653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060e540-71ab-4367-95d2-8f95a6e769db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.599440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1060e540-71ab-4367-95d2-8f95a6e769db None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:25.600576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1060e540-71ab-4367-95d2-8f95a6e769db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:25.615085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:25.621849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa359c-f8e4-4c5a-b6b3-d5b1d11d7fdb None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:25.658479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:25.659282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.659487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.659766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] PATH_INFO: `/v3/users/75d0ac7f45544538b4eb18abbe65432f` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.673480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] RBAC: Authorizing `identity:get_user(user_id=75d0ac7f45544538b4eb18abbe65432f)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:25.675598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:25.676595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 542/1057] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:25.681946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 516/1058] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:25 2022] GET /identity/v3/users/75d0ac7f45544538b4eb18abbe65432f => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:25.685524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060e540-71ab-4367-95d2-8f95a6e769db None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:25.698817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:25.771484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:25.795090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060e540-71ab-4367-95d2-8f95a6e769db None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:25.810367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0590e0d-a060-452a-a71f-d28f9bbd0600 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:25.829817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82ea55-5071-4209-a1d5-3265612e39b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:25.830892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ea55-5071-4209-a1d5-3265612e39b7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.831026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ea55-5071-4209-a1d5-3265612e39b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.831279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ea55-5071-4209-a1d5-3265612e39b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.847222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3029ad-9268-413a-ac6f-f54667c9a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:25.847865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3029ad-9268-413a-ac6f-f54667c9a2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.848149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3029ad-9268-413a-ac6f-f54667c9a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.848429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3029ad-9268-413a-ac6f-f54667c9a2f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.848881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:25.858725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3029ad-9268-413a-ac6f-f54667c9a2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-db3029ad-9268-413a-ac6f-f54667c9a2f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:25.864202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:25.865302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:25.866143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 517/1059] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 29 06:14:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:25.866849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466365d9-463e-4f64-9135-51883146e32c None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:25.867092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466365d9-463e-4f64-9135-51883146e32c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:25.867360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466365d9-463e-4f64-9135-51883146e32c None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:25.868604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 518/1060] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:25.871150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466365d9-463e-4f64-9135-51883146e32c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:25.904456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82ea55-5071-4209-a1d5-3265612e39b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:25.905939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82ea55-5071-4209-a1d5-3265612e39b7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:25.972134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466365d9-463e-4f64-9135-51883146e32c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:25.995132 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.007958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.008689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.008975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.009251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.009464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 543/1061] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:26.009964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:26.010291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.010716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dc1fa-e40c-4eed-8e05-5c60d63039b3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.011003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dc1fa-e40c-4eed-8e05-5c60d63039b3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.011282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dc1fa-e40c-4eed-8e05-5c60d63039b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.011706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.020037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 519/1062] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 29 06:14:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:26.022252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.031818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-740dc1fa-e40c-4eed-8e05-5c60d63039b3 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:26.033038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-740dc1fa-e40c-4eed-8e05-5c60d63039b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:26.077519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.102945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 544/1063] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:26.103989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dc1fa-e40c-4eed-8e05-5c60d63039b3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.130752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f83c5-cbe7-426c-ba80-bfd0e12421e6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.164693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.165343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.165625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.165907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.167052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] RBAC: Authorizing `identity:get_project(project_id=30c681ce58b24f419ebc80c295154b6f)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.168465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.170147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 520/1064] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:26.170901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.209482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dc1fa-e40c-4eed-8e05-5c60d63039b3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.222729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.244335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.245006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.245287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.245560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.255556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.257022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.270384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15230ff6-68cd-427d-9e6c-4efb93422da6 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:26.271879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:26.272864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:26.273783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:26.273783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:26.273783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:26.273783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:26.273783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 545/1065] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:26.274104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.304819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.305318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.305947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.306231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.306509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.375009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33af57-fbff-484d-ace2-3b68042b79d7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.375235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.376478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.408528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.409127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.409410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.409692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.410472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=30c681ce58b24f419ebc80c295154b6f, user.id=75d0ac7f45544538b4eb18abbe65432f) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:26.410784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.412202 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.417507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 546/1066] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/role_assignments?user.id=75d0ac7f45544538b4eb18abbe65432f&scope.project.id=30c681ce58b24f419ebc80c295154b6f&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:26.418278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.432532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.489397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 521/1067] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:26.495460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.524458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.532786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06353539-4ded-4c0e-aa8b-e7340b0d4d5b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.569688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.570320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.570603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.570879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.571565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:26.572204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.573421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.591029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 547/1068] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:26.592565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.609753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fb6be-42f5-47a4-aee5-d5912fed6129 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.649292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.650244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.650533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.650809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.661933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.663311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:26.678797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:26.679717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:26.680648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:26.680648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:26.680648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:26.680648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:26.680648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 522/1069] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:26.685702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.690912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.707643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316edf7-e238-4576-b431-4c790c4fbabf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.744349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.745044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.745328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.745608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.757011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.758495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:14:26.770968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:26.771967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:26.772988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 548/1070] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:26.777846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:26.794732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:26.805998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484e63e-4102-4824-9c79-01ca36caa733 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.843485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.844115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.844385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.844698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.845380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:26.846224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.847725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.860033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 523/1071] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:26.887108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196d750-08c7-4059-b23b-5dde52bc63d8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:26.928298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:26.928908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:26.929256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:26.929544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:26.930438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:26.930758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:26.932337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:26.953264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 549/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:26.959160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:27.028552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:27.072205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727eb19-9e4b-4552-9739-23aacd2a8f37 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:27.107948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:27.108626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:27.108903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:27.109177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:27.119828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:27.121286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:27.137627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:27.138976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:27.139862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:27.139862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:27.139862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:27.139862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:27.139862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 524/1073] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:27.145368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:27.217779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:27.247930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e208e1-0840-43e6-9d96-ccbb0ad7589a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:27.281298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:27.281914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:27.282200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:27.282480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:27.283753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:27.284067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:27.285461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:27.296493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 550/1074] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:27.302060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:27.400325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d62316-6511-4009-a9be-a8b603e8a3a8 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:27.416471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:27.435534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:27.436181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:27.436467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:27.436777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:27.443290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:27.444755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:27.457004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:27.458004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:27.458987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 525/1075] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:27.463914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:27.567940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573a8ca-b21c-4978-a1fb-895f32955411 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:27.595511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:27.601867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099127d-6be6-43e4-ba41-908dfbf2f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:27.602481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099127d-6be6-43e4-ba41-908dfbf2f89e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:27.602762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099127d-6be6-43e4-ba41-908dfbf2f89e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:27.603035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099127d-6be6-43e4-ba41-908dfbf2f89e None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:27.603763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099127d-6be6-43e4-ba41-908dfbf2f89e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:27.604067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099127d-6be6-43e4-ba41-908dfbf2f89e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:27.605507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099127d-6be6-43e4-ba41-908dfbf2f89e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:27.617226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 551/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:27.623047 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:27.719371 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c03fb9-5276-4636-b759-c4e8a356f6aa None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:27.755390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecaa68-7937-4d92-9d2d-b7a313b76c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:27.756117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecaa68-7937-4d92-9d2d-b7a313b76c4f None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:27.756400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecaa68-7937-4d92-9d2d-b7a313b76c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:27.756706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecaa68-7937-4d92-9d2d-b7a313b76c4f None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/bbe64416672b408f8cdf93220cc31ef5/roles/ab91e6fc0e6942c2bd23a93741d5b1d8` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:27.758825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecaa68-7937-4d92-9d2d-b7a313b76c4f None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=bbe64416672b408f8cdf93220cc31ef5, role_id=ab91e6fc0e6942c2bd23a93741d5b1d8)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:27.760485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecaa68-7937-4d92-9d2d-b7a313b76c4f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:27.773358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 526/1077] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:14:27 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/bbe64416672b408f8cdf93220cc31ef5/roles/ab91e6fc0e6942c2bd23a93741d5b1d8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:14:28.422200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940a12-7215-4f30-8cdf-634471f637d0 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:28.422780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940a12-7215-4f30-8cdf-634471f637d0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:28.422895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940a12-7215-4f30-8cdf-634471f637d0 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:28.425118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 552/1078] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:28.430381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6a79a-fd45-458f-af8a-cd91310d4468 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:28.430825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6a79a-fd45-458f-af8a-cd91310d4468 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:28.430919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6a79a-fd45-458f-af8a-cd91310d4468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:28.448909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-27f6a79a-fd45-458f-af8a-cd91310d4468 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:28.450203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-27f6a79a-fd45-458f-af8a-cd91310d4468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:28.478688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:28.518384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 527/1079] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:28.540848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940a12-7215-4f30-8cdf-634471f637d0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:28.646786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940a12-7215-4f30-8cdf-634471f637d0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:28.680111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:28.680712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:28.681001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:28.681282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:28.691127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:28.696029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:28.697027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:28.697867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 553/1080] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:28.703218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6a79a-fd45-458f-af8a-cd91310d4468 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:28.804148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6a79a-fd45-458f-af8a-cd91310d4468 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:28.838445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:28.839076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:28.839417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:28.839702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:28.840414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:28.840756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:28.842117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:28.850966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 528/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:28.856774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:28.911409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:28.958178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf0299-4917-4a0d-a593-2b3c84bf14af None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:28.990812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:28.991508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:28.991807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:28.992066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] PATH_INFO: `/v3/users/75d0ac7f45544538b4eb18abbe65432f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:28.993221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] RBAC: Authorizing `identity:get_user(user_id=75d0ac7f45544538b4eb18abbe65432f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:28.994639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:28.996317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 554/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:28 2022] GET /identity/v3/users/75d0ac7f45544538b4eb18abbe65432f => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:29.002735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:29.031011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:29.102742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a86eb0-33bb-4800-acdf-7c53c20ac401 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:29.135583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242966e0-0e5c-45e4-946a-00ce03195b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:29.136216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242966e0-0e5c-45e4-946a-00ce03195b59 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:29.136500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242966e0-0e5c-45e4-946a-00ce03195b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:29.136808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242966e0-0e5c-45e4-946a-00ce03195b59 None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:29.137958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242966e0-0e5c-45e4-946a-00ce03195b59 None admin] RBAC: Authorizing `identity:get_project(project_id=30c681ce58b24f419ebc80c295154b6f)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:29.139373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242966e0-0e5c-45e4-946a-00ce03195b59 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:29.141057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 529/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:29 2022] GET /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:29.147596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:29.244837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48f983-0568-41e7-8999-8229a3975f81 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:29.265581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:29.280902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554e4b5-dfc4-4bb3-b97f-f1f3d9e8208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:29.281578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554e4b5-dfc4-4bb3-b97f-f1f3d9e8208e None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:29.281863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554e4b5-dfc4-4bb3-b97f-f1f3d9e8208e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:29.282143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554e4b5-dfc4-4bb3-b97f-f1f3d9e8208e None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f/users/75d0ac7f45544538b4eb18abbe65432f/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:29.284231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554e4b5-dfc4-4bb3-b97f-f1f3d9e8208e None admin] RBAC: Authorizing `identity:create_grant(project_id=30c681ce58b24f419ebc80c295154b6f, user_id=75d0ac7f45544538b4eb18abbe65432f, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:29.285697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554e4b5-dfc4-4bb3-b97f-f1f3d9e8208e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:29.298255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 555/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:14:29 2022] PUT /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f/users/75d0ac7f45544538b4eb18abbe65432f/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:14:29.766059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45134583-cc86-4716-a990-a66fac478f10 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:29.766565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45134583-cc86-4716-a990-a66fac478f10 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:29.766655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45134583-cc86-4716-a990-a66fac478f10 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:29.768577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 530/1085] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:29.773774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bf758-1e4a-43a1-a6f1-6cb7b979c479 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:29.774201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bf758-1e4a-43a1-a6f1-6cb7b979c479 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:29.774321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bf758-1e4a-43a1-a6f1-6cb7b979c479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:29.792090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1e2bf758-1e4a-43a1-a6f1-6cb7b979c479 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:29.793299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e2bf758-1e4a-43a1-a6f1-6cb7b979c479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:29.858124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 556/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:29.882513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45134583-cc86-4716-a990-a66fac478f10 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:29.978398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:29.986101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45134583-cc86-4716-a990-a66fac478f10 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:30.021893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:30.022568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:30.022855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:30.023129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:30.033592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:30.038456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:30.039569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 06:14:30.040656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:30.044596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 531/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 29 06:14:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:30.045587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bf758-1e4a-43a1-a6f1-6cb7b979c479 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:30.142255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bf758-1e4a-43a1-a6f1-6cb7b979c479 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:30.153248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:30.177638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:30.178253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:30.178540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:30.178820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:30.179522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:30.179831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:30.181232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:30.188930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 557/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 29 06:14:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:30.194485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:30.292982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebf46c-b95c-4dbb-b51b-f6d2daf65b10 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:30.325385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:30.326025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:30.326308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:30.326585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:30.337834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:30.339309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:30.354506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:30.355576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:30.356491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:30.356491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:30.356491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:30.356491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:30.356491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 532/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:30.361477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:30.463973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb8744-f73d-431d-aa24-228e9a8712f9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:30.497700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:30.498320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:30.498604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:30.498879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:30.499556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:30.499863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:30.501259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:30.513386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 558/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:30.519571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:30.531369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:30.618126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd7525-8f20-4c38-a1c0-875899720950 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:30.652991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:30.653632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:30.653917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:30.654196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:30.661782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:30.663208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:14:30.674250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:30.675260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 06:14:30.676246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:30.676836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 533/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 29 06:14:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:30.681064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:30.712698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:30.781857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64808c2-802b-4673-b873-8fddb7631081 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:30.816871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:30.817491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:30.817778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:30.818055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:30.818836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:30.819146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:30.820589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:30.837799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 559/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 29 06:14:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:30.843589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:30.909696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:30.944332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb8b48-0ac0-4d82-a05d-72276e51f46a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:30.979408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054caae6-7d71-47f6-955e-c2e10d02257c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:30.980031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054caae6-7d71-47f6-955e-c2e10d02257c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:30.980319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054caae6-7d71-47f6-955e-c2e10d02257c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:30.980629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054caae6-7d71-47f6-955e-c2e10d02257c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.051316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054caae6-7d71-47f6-955e-c2e10d02257c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.052778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054caae6-7d71-47f6-955e-c2e10d02257c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.115492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.149721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 534/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 06:14:31.155567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.243897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.254157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e96ef-95b8-452f-90b4-26541c48b9f3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.271478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73462a-d0c0-4917-b15a-d947171c2420 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.271772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73462a-d0c0-4917-b15a-d947171c2420 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.272052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73462a-d0c0-4917-b15a-d947171c2420 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.273433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 535/1094] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:14:31.278214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64a105-7b12-477c-a9ba-54ba82eff530 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.278505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64a105-7b12-477c-a9ba-54ba82eff530 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.278792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64a105-7b12-477c-a9ba-54ba82eff530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.289302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.289944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.290228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.290505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.296891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9c64a105-7b12-477c-a9ba-54ba82eff530 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:31.298077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9c64a105-7b12-477c-a9ba-54ba82eff530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:31.301241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.302683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:14:31.317251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:31.318294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:31.319187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:14:31.319187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:14:31.319187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:14:31.319187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:31.319187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 560/1095] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:31.323968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.371426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 536/1096] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:31.397441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64a105-7b12-477c-a9ba-54ba82eff530 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.423312 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2ce85-1bc7-4ead-9402-e76e372b067c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.458435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.459056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.459335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.459622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.460306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:31.460636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.462009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.473469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 561/1097] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:31.478967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.481537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.493079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.496783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64a105-7b12-477c-a9ba-54ba82eff530 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.530745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.531394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.531688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.531967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.542539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:31.547340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:31.548309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:31.549349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 537/1098] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:31.554447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.577594 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654e0d-ce90-43ba-94b8-808cccc46eaa None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.612678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.613325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.613608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.613887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.620353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.621857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:14:31.635405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:31.636380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:31.637333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 562/1099] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:14:31.642237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.653519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90b6b9-bcb7-4a88-8d1f-32a1bf13c675 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.689520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.690139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.690424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.690701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.691411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:31.691718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.693116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.701121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 538/1100] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:31.707988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.708116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.730001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.748214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd18a15-53cb-4240-97bc-6bc439e7a0d2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.781450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.782063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.782590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.782590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.783969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:31.784274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.785745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.797605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 563/1101] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:31.803571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.809405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb520d1f-13d0-4225-bc08-61a807f6bb9b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.842577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.843213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.843493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.843766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] PATH_INFO: `/v3/users/75d0ac7f45544538b4eb18abbe65432f` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.844952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] RBAC: Authorizing `identity:get_user(user_id=75d0ac7f45544538b4eb18abbe65432f)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.846414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.848004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 539/1102] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/users/75d0ac7f45544538b4eb18abbe65432f => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:31.853790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:31.907374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41e1d7-0520-48fd-ac4a-80790f1ba1d5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.933845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.941695 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:31.946599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:31.947219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:31.947503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:31.947779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:31.948581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] RBAC: Adding query filter params (role.id=ab91e6fc0e6942c2bd23a93741d5b1d8, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=bbe64416672b408f8cdf93220cc31ef5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:31.948898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:31.950325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:31.954059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f2b14-eabd-4e49-9c74-ee235b55aadb None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:31.957203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 564/1103] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/role_assignments?user.id=bbe64416672b408f8cdf93220cc31ef5&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=ab91e6fc0e6942c2bd23a93741d5b1d8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:32.010139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:32.010710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:32.010996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:32.011276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:32.085009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:32.086483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:32.174703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:32.187569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 540/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 06:14:32.194291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:32.297788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07018952-490e-43c6-bb6a-0197e4f54ca6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:32.334393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044446cb-954b-43ed-bde0-3a1385e55d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:32.335040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044446cb-954b-43ed-bde0-3a1385e55d3d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:32.335320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044446cb-954b-43ed-bde0-3a1385e55d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:32.335594 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044446cb-954b-43ed-bde0-3a1385e55d3d None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:32.336776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044446cb-954b-43ed-bde0-3a1385e55d3d None admin] RBAC: Authorizing `identity:get_project(project_id=30c681ce58b24f419ebc80c295154b6f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:32.338209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044446cb-954b-43ed-bde0-3a1385e55d3d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:32.339873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 565/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:32 2022] GET /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:32.346666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:32.449824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8acd75-d4af-4779-a210-5afd580daee0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:32.484430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7284721-c306-457b-b6fb-6d58c58687f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:32.485086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7284721-c306-457b-b6fb-6d58c58687f1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:32.485367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7284721-c306-457b-b6fb-6d58c58687f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:32.485640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7284721-c306-457b-b6fb-6d58c58687f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:32.487245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7284721-c306-457b-b6fb-6d58c58687f1 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=30c681ce58b24f419ebc80c295154b6f, user.id=75d0ac7f45544538b4eb18abbe65432f) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:32.487245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7284721-c306-457b-b6fb-6d58c58687f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:32.488693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7284721-c306-457b-b6fb-6d58c58687f1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:32.494597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 541/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:32 2022] GET /identity/v3/role_assignments?user.id=75d0ac7f45544538b4eb18abbe65432f&scope.project.id=30c681ce58b24f419ebc80c295154b6f&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:34.525579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ef68-ef26-4905-a6de-ea8c664e85d4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:34.526143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ef68-ef26-4905-a6de-ea8c664e85d4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:34.526143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ef68-ef26-4905-a6de-ea8c664e85d4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:34.527460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 566/1107] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:34.532495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca94f8f-6fcd-4204-8d0c-5bcd6a60fb7a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:34.533435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca94f8f-6fcd-4204-8d0c-5bcd6a60fb7a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:34.533679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca94f8f-6fcd-4204-8d0c-5bcd6a60fb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:34.552499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5ca94f8f-6fcd-4204-8d0c-5bcd6a60fb7a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:34.553714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5ca94f8f-6fcd-4204-8d0c-5bcd6a60fb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:34.599400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:34.619747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 542/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:34.643937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ef68-ef26-4905-a6de-ea8c664e85d4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:34.704310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:34.749228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ef68-ef26-4905-a6de-ea8c664e85d4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:34.781918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea4a4e-c5cd-486d-a0d0-be799ed9056f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:34.782486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea4a4e-c5cd-486d-a0d0-be799ed9056f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:34.782779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea4a4e-c5cd-486d-a0d0-be799ed9056f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:34.783251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea4a4e-c5cd-486d-a0d0-be799ed9056f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:34.784388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea4a4e-c5cd-486d-a0d0-be799ed9056f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:34.785838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea4a4e-c5cd-486d-a0d0-be799ed9056f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:34.787397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 567/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:34.793180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca94f8f-6fcd-4204-8d0c-5bcd6a60fb7a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:34.888829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca94f8f-6fcd-4204-8d0c-5bcd6a60fb7a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:34.922189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff125c-3685-46a9-90ab-d93ede04c48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:34.922801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff125c-3685-46a9-90ab-d93ede04c48c None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:34.923114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff125c-3685-46a9-90ab-d93ede04c48c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:34.923425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff125c-3685-46a9-90ab-d93ede04c48c None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:34.924847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff125c-3685-46a9-90ab-d93ede04c48c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:34.926265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff125c-3685-46a9-90ab-d93ede04c48c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:34.947024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 543/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:14:34 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:36.971874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746090d-ea76-43ac-97bf-0b4278c928e8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:36.972431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746090d-ea76-43ac-97bf-0b4278c928e8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:36.972431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746090d-ea76-43ac-97bf-0b4278c928e8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:36.973840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 568/1111] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:36.979001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0e856-1624-4046-b080-697fb9319f46 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:36.979443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0e856-1624-4046-b080-697fb9319f46 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:36.979553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0e856-1624-4046-b080-697fb9319f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:36.997157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-03e0e856-1624-4046-b080-697fb9319f46 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:36.998262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-03e0e856-1624-4046-b080-697fb9319f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:37.026866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:37.063407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 544/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:37.086618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746090d-ea76-43ac-97bf-0b4278c928e8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:37.185623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746090d-ea76-43ac-97bf-0b4278c928e8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:37.208229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:37.219689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:37.220997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:37.221286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:37.221562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] PATH_INFO: `/v3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:37.222998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] RBAC: Authorizing `identity:get_role(role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:37.224352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:37.225713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 569/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:37 2022] GET /identity/v3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:37.231442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0e856-1624-4046-b080-697fb9319f46 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:37.326496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0e856-1624-4046-b080-697fb9319f46 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:37.359654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:37.360284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:37.360586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:37.360888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] PATH_INFO: `/v3/users/b1d7a4371221406c9145e561000de199` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:37.373341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] RBAC: Authorizing `identity:get_user(user_id=b1d7a4371221406c9145e561000de199)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:37.374754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:37.376327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 545/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:37 2022] GET /identity/v3/users/b1d7a4371221406c9145e561000de199 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:37.382531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:37.479724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c71b2d-27f2-4c37-8074-e77f3bbb1641 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:37.513103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:37.513709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:37.513995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:37.514273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:37.569704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:37.580825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:37.582213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:37.677565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 570/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:37.683369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:37.710159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:37.783756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d121c0a-4538-4699-b157-b612e4f6e172 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:37.815895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfa4f8-4592-48ff-bf11-076d25d760fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:37.816526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa4f8-4592-48ff-bf11-076d25d760fa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:37.816837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa4f8-4592-48ff-bf11-076d25d760fa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:37.817109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa4f8-4592-48ff-bf11-076d25d760fa None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:37.818242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfa4f8-4592-48ff-bf11-076d25d760fa None admin] RBAC: Authorizing `identity:get_project(project_id=30c681ce58b24f419ebc80c295154b6f)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:37.819643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfa4f8-4592-48ff-bf11-076d25d760fa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:37.821319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 546/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:37 2022] GET /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:37.827393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:37.919692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:37.929778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68865fea-ec3d-479e-9a71-0c927457963e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:37.963157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d065ae0-5c2f-4ac8-8cf3-388e92bff79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:37.963759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d065ae0-5c2f-4ac8-8cf3-388e92bff79a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:37.964039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d065ae0-5c2f-4ac8-8cf3-388e92bff79a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:37.964311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d065ae0-5c2f-4ac8-8cf3-388e92bff79a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:37.965164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d065ae0-5c2f-4ac8-8cf3-388e92bff79a None admin] RBAC: Adding query filter params (role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=30c681ce58b24f419ebc80c295154b6f, user.id=b1d7a4371221406c9145e561000de199) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:37.965472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d065ae0-5c2f-4ac8-8cf3-388e92bff79a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:37.966879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d065ae0-5c2f-4ac8-8cf3-388e92bff79a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:37.972685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 571/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:37 2022] GET /identity/v3/role_assignments?user.id=b1d7a4371221406c9145e561000de199&scope.project.id=30c681ce58b24f419ebc80c295154b6f&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:39.926627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7bf79-804e-4999-846e-9c220f4478c6 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:39.927202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7bf79-804e-4999-846e-9c220f4478c6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:39.927202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7bf79-804e-4999-846e-9c220f4478c6 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:39.928569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 547/1118] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:39.933590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeece8-59dd-433b-bacc-0a0ac301055e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:39.933995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeece8-59dd-433b-bacc-0a0ac301055e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:39.934148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeece8-59dd-433b-bacc-0a0ac301055e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:39.951984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e7aeece8-59dd-433b-bacc-0a0ac301055e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:39.953293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e7aeece8-59dd-433b-bacc-0a0ac301055e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:40.017930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 572/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:40.039897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7bf79-804e-4999-846e-9c220f4478c6 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:40.131443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:40.139165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7bf79-804e-4999-846e-9c220f4478c6 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:40.173530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:40.174160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:40.174445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:40.174726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] PATH_INFO: `/v3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:40.176163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] RBAC: Authorizing `identity:get_role(role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:40.177560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:40.178890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 548/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:40 2022] GET /identity/v3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:40.184648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeece8-59dd-433b-bacc-0a0ac301055e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:40.267038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:40.283644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeece8-59dd-433b-bacc-0a0ac301055e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:40.316017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:40.316669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:40.317081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:40.317362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] PATH_INFO: `/v3/users/b1d7a4371221406c9145e561000de199` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:40.318499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] RBAC: Authorizing `identity:get_user(user_id=b1d7a4371221406c9145e561000de199)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:40.319913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:40.321510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 573/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:40 2022] GET /identity/v3/users/b1d7a4371221406c9145e561000de199 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:40.327116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:40.423224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d3314-01de-4504-9e31-83aa863a479d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:40.456099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6be9e-266a-47c6-b126-62f9c9ee1897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:40.456761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6be9e-266a-47c6-b126-62f9c9ee1897 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:40.457047 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6be9e-266a-47c6-b126-62f9c9ee1897 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:40.457327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6be9e-266a-47c6-b126-62f9c9ee1897 None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:40.458469 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6be9e-266a-47c6-b126-62f9c9ee1897 None admin] RBAC: Authorizing `identity:get_project(project_id=30c681ce58b24f419ebc80c295154b6f)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:40.459885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6be9e-266a-47c6-b126-62f9c9ee1897 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:40.461553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 549/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:40 2022] GET /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:40.467283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:40.563670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a70-8aa9-4f87-8d44-3f2e7a43429f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:40.597618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d052d-7fe5-46d1-b9b8-b81dc43639eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:40.598277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d052d-7fe5-46d1-b9b8-b81dc43639eb None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:40.598563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d052d-7fe5-46d1-b9b8-b81dc43639eb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:40.598845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d052d-7fe5-46d1-b9b8-b81dc43639eb None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f/users/b1d7a4371221406c9145e561000de199/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:40.601028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d052d-7fe5-46d1-b9b8-b81dc43639eb None admin] RBAC: Authorizing `identity:create_grant(project_id=30c681ce58b24f419ebc80c295154b6f, user_id=b1d7a4371221406c9145e561000de199, role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:40.602457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d052d-7fe5-46d1-b9b8-b81dc43639eb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:40.615774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 574/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:14:40 2022] PUT /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f/users/b1d7a4371221406c9145e561000de199/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:14:42.545660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861c1e-b2a2-4152-b9c6-ac9843d1567c None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:42.546219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861c1e-b2a2-4152-b9c6-ac9843d1567c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:42.546219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861c1e-b2a2-4152-b9c6-ac9843d1567c None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:42.547544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 550/1124] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:42.553109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c520bf-d913-4805-a549-ae11e030d5a5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:42.553566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c520bf-d913-4805-a549-ae11e030d5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:42.553668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c520bf-d913-4805-a549-ae11e030d5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:42.571387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e4c520bf-d913-4805-a549-ae11e030d5a5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:42.572565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e4c520bf-d913-4805-a549-ae11e030d5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:42.579748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:42.638645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 575/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:42.661741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861c1e-b2a2-4152-b9c6-ac9843d1567c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:42.691666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:42.764973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861c1e-b2a2-4152-b9c6-ac9843d1567c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:42.797707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:42.798827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:42.799112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:42.799381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] PATH_INFO: `/v3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:42.800836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] RBAC: Authorizing `identity:get_role(role_id=4341e45b8e3c4314a4a9483cdeeb9cc1)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:42.802185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:42.803521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 551/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:42 2022] GET /identity/v3/roles/4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:42.809421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c520bf-d913-4805-a549-ae11e030d5a5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:42.905868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c520bf-d913-4805-a549-ae11e030d5a5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:42.938364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:42.939099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:42.939380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:42.939652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] PATH_INFO: `/v3/users/b1d7a4371221406c9145e561000de199` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:42.940824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=b1d7a4371221406c9145e561000de199)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:42.942332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:42.943974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 576/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:42 2022] GET /identity/v3/users/b1d7a4371221406c9145e561000de199 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:42.949527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:43.041939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:43.045508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b1fce-9469-4d93-9c9c-4ea21f43c106 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:43.077704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:43.078248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:43.078527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:43.078800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:43.143570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:43.145020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:43.234718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:43.239862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 552/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:43.245647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:43.258347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:43.344655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd16f3-b045-4f43-95fd-6c8ebfebe4a1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:43.377866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f837318-301b-4a67-8c3d-ecf301d886af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:43.378494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f837318-301b-4a67-8c3d-ecf301d886af None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:43.378774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f837318-301b-4a67-8c3d-ecf301d886af None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:43.379047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f837318-301b-4a67-8c3d-ecf301d886af None admin] PATH_INFO: `/v3/projects/30c681ce58b24f419ebc80c295154b6f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:43.380174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f837318-301b-4a67-8c3d-ecf301d886af None admin] RBAC: Authorizing `identity:get_project(project_id=30c681ce58b24f419ebc80c295154b6f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:43.381621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f837318-301b-4a67-8c3d-ecf301d886af None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:43.383258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 577/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:43 2022] GET /identity/v3/projects/30c681ce58b24f419ebc80c295154b6f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:43.389272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:43.484059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe826a-4aa7-4bf3-ad48-692d27a3a205 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:43.517835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b5113-742e-40de-93bd-edfab76a18cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:43.518479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b5113-742e-40de-93bd-edfab76a18cb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:43.518760 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b5113-742e-40de-93bd-edfab76a18cb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:43.519036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b5113-742e-40de-93bd-edfab76a18cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:43.519797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b5113-742e-40de-93bd-edfab76a18cb None admin] RBAC: Adding query filter params (role.id=4341e45b8e3c4314a4a9483cdeeb9cc1, scope.project.id=30c681ce58b24f419ebc80c295154b6f, user.id=b1d7a4371221406c9145e561000de199) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:43.520104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b5113-742e-40de-93bd-edfab76a18cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:43.521531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b5113-742e-40de-93bd-edfab76a18cb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:43.527114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 553/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:43 2022] GET /identity/v3/role_assignments?user.id=b1d7a4371221406c9145e561000de199&scope.project.id=30c681ce58b24f419ebc80c295154b6f&role.id=4341e45b8e3c4314a4a9483cdeeb9cc1 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:14:45.531072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737c9ab-d3ee-4325-89d8-127033f40cd2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:45.531632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737c9ab-d3ee-4325-89d8-127033f40cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:45.531632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737c9ab-d3ee-4325-89d8-127033f40cd2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:45.533501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 578/1131] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:45.539242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fff9b-57c3-446a-b52a-88d83aacd62c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:45.539644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fff9b-57c3-446a-b52a-88d83aacd62c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:45.539803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fff9b-57c3-446a-b52a-88d83aacd62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:45.558683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-866fff9b-57c3-446a-b52a-88d83aacd62c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:45.559867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-866fff9b-57c3-446a-b52a-88d83aacd62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:45.614601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:45.625059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 554/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:45.648129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737c9ab-d3ee-4325-89d8-127033f40cd2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:45.725632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:45.748839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737c9ab-d3ee-4325-89d8-127033f40cd2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:45.781662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4edbf-76a4-4b12-a636-0fc3f8ea83a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:45.782234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4edbf-76a4-4b12-a636-0fc3f8ea83a0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:45.783145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4edbf-76a4-4b12-a636-0fc3f8ea83a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:45.783436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4edbf-76a4-4b12-a636-0fc3f8ea83a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:45.784617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4edbf-76a4-4b12-a636-0fc3f8ea83a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:45.786049 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4edbf-76a4-4b12-a636-0fc3f8ea83a0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:45.787635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 579/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:45.793446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fff9b-57c3-446a-b52a-88d83aacd62c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:45.888291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fff9b-57c3-446a-b52a-88d83aacd62c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:45.921486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b3029-533a-4469-a985-ce7c8f17f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:45.922095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3029-533a-4469-a985-ce7c8f17f275 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:45.922375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3029-533a-4469-a985-ce7c8f17f275 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:45.922649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3029-533a-4469-a985-ce7c8f17f275 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:45.923364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b3029-533a-4469-a985-ce7c8f17f275 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:45.924792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b3029-533a-4469-a985-ce7c8f17f275 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:45.950151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 555/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:14:45 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:14:48.143108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9a83-f6c1-4f4b-83d8-4cecc94e2c10 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:48.143653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9a83-f6c1-4f4b-83d8-4cecc94e2c10 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:48.143653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9a83-f6c1-4f4b-83d8-4cecc94e2c10 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:48.145039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 580/1135] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:48.150086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf1e3f-955e-4cca-bea4-d81c7717db62 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:48.150486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf1e3f-955e-4cca-bea4-d81c7717db62 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:48.150641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf1e3f-955e-4cca-bea4-d81c7717db62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:48.168773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7abf1e3f-955e-4cca-bea4-d81c7717db62 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:48.169944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7abf1e3f-955e-4cca-bea4-d81c7717db62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:48.209138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:48.236830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 556/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:48.260300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e9a83-f6c1-4f4b-83d8-4cecc94e2c10 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:48.360221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e9a83-f6c1-4f4b-83d8-4cecc94e2c10 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:48.393381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78d31f-00df-473b-9ec7-79258d47995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:48.393956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78d31f-00df-473b-9ec7-79258d47995f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:48.394242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78d31f-00df-473b-9ec7-79258d47995f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:48.394519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78d31f-00df-473b-9ec7-79258d47995f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:48.395659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78d31f-00df-473b-9ec7-79258d47995f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:48.397122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78d31f-00df-473b-9ec7-79258d47995f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:48.398700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 581/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:48.404532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf1e3f-955e-4cca-bea4-d81c7717db62 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:48.500892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf1e3f-955e-4cca-bea4-d81c7717db62 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:48.534800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73d257-87bd-4559-adfd-738c3ea29baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:48.535416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d257-87bd-4559-adfd-738c3ea29baf None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:48.535699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d257-87bd-4559-adfd-738c3ea29baf None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:48.535976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d257-87bd-4559-adfd-738c3ea29baf None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:48.536694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73d257-87bd-4559-adfd-738c3ea29baf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:48.538105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73d257-87bd-4559-adfd-738c3ea29baf None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:48.559034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 557/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:14:48 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:14:50.583974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca42d-e019-45e8-83f0-913aaa3d02e5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:50.584575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca42d-e019-45e8-83f0-913aaa3d02e5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:50.584575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca42d-e019-45e8-83f0-913aaa3d02e5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:50.585888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 582/1139] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:50.591032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92147a31-58fa-46b1-8de5-f2ab39c89b1e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:50.591451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92147a31-58fa-46b1-8de5-f2ab39c89b1e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:50.591577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92147a31-58fa-46b1-8de5-f2ab39c89b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:50.609849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-92147a31-58fa-46b1-8de5-f2ab39c89b1e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:50.611142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-92147a31-58fa-46b1-8de5-f2ab39c89b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:50.633953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:50.678462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 558/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:50.701505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca42d-e019-45e8-83f0-913aaa3d02e5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:50.720693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:50.803696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca42d-e019-45e8-83f0-913aaa3d02e5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:50.836520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:50.837113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:50.837397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:50.837674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:50.848104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:50.853008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:50.854196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:50.855327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 583/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:14:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:50.860318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92147a31-58fa-46b1-8de5-f2ab39c89b1e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:50.957517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92147a31-58fa-46b1-8de5-f2ab39c89b1e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:50.990380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:50.990999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:50.991286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:50.991567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:50.992274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:50.992608 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:50.993964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:51.002546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 559/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:14:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:51.008256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:51.082358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:51.106794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db30a83-58b9-4fbf-b260-24971a5b4288 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:51.139577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:51.140203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:51.140480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:51.140782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] PATH_INFO: `/v3/users/73a448832c8e459495eccd60bac62759` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:51.153899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] RBAC: Authorizing `identity:get_user(user_id=73a448832c8e459495eccd60bac62759)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:51.155308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:51.156935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 584/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:51 2022] GET /identity/v3/users/73a448832c8e459495eccd60bac62759 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:51.163441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:51.188559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:51.260458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa2d02-2f0b-47c3-a5cf-d17c7bce92f0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:51.293013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:51.293611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:51.293889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:51.294163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:51.358845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:51.360199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:51.381813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:51.456023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 560/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:51.461799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:51.561706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b80e34-a2ee-4d6c-a627-4355ccc778f8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:51.590335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:51.596104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5f78-2f96-49bc-a4cd-8244416c2253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:51.596773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5f78-2f96-49bc-a4cd-8244416c2253 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:51.597060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5f78-2f96-49bc-a4cd-8244416c2253 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:51.597337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5f78-2f96-49bc-a4cd-8244416c2253 None admin] PATH_INFO: `/v3/projects/4d7df929a74a4354a080ad68279e6d4b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:51.598488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5f78-2f96-49bc-a4cd-8244416c2253 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7df929a74a4354a080ad68279e6d4b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:51.600074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5f78-2f96-49bc-a4cd-8244416c2253 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:51.602101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 585/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:51 2022] GET /identity/v3/projects/4d7df929a74a4354a080ad68279e6d4b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:51.608180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:51.703850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4cd68-cb84-434d-af20-fac631212d8e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:51.726222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:51.739339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a25b20-5dc9-4191-b6de-b3e34bf73efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:51.739957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a25b20-5dc9-4191-b6de-b3e34bf73efa None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:51.740234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a25b20-5dc9-4191-b6de-b3e34bf73efa None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:51.740510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a25b20-5dc9-4191-b6de-b3e34bf73efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:51.741312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a25b20-5dc9-4191-b6de-b3e34bf73efa None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=4d7df929a74a4354a080ad68279e6d4b, user.id=73a448832c8e459495eccd60bac62759) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:51.741621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a25b20-5dc9-4191-b6de-b3e34bf73efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:51.743022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a25b20-5dc9-4191-b6de-b3e34bf73efa None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:51.748259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 561/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:51 2022] GET /identity/v3/role_assignments?user.id=73a448832c8e459495eccd60bac62759&scope.project.id=4d7df929a74a4354a080ad68279e6d4b&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:53.711498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd92af-33f7-4c13-8671-254bccb69a7e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:53.712070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd92af-33f7-4c13-8671-254bccb69a7e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:53.712070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd92af-33f7-4c13-8671-254bccb69a7e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:53.713505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 586/1147] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:53.718501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab176f-35b5-4fa1-b14f-e7760cfb2072 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:53.718917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab176f-35b5-4fa1-b14f-e7760cfb2072 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:53.719064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab176f-35b5-4fa1-b14f-e7760cfb2072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:53.737379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3cab176f-35b5-4fa1-b14f-e7760cfb2072 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:53.738544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3cab176f-35b5-4fa1-b14f-e7760cfb2072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:53.802329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 562/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:14:53.824521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd92af-33f7-4c13-8671-254bccb69a7e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:53.922520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd92af-33f7-4c13-8671-254bccb69a7e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:53.955658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:53.956283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:53.956582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:53.956860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:53.966541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:53.971436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:53.972564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:14:53.973442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 587/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:14:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:53.978565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab176f-35b5-4fa1-b14f-e7760cfb2072 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:54.074243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab176f-35b5-4fa1-b14f-e7760cfb2072 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:54.090940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:54.108296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:54.108941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:54.109229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:54.109507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:54.110739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:54.111046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:54.112494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:54.120306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 563/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:14:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:54.126596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:54.184275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:54.224618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48a319-26ba-40e0-b728-7533dca66886 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:54.257664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:54.258292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:54.258576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:54.258853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] PATH_INFO: `/v3/users/73a448832c8e459495eccd60bac62759` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:54.260003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] RBAC: Authorizing `identity:get_user(user_id=73a448832c8e459495eccd60bac62759)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:54.261457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:54.263032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 588/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:54 2022] GET /identity/v3/users/73a448832c8e459495eccd60bac62759 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:54.269445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:54.366315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f93e8-63b3-4a3f-9091-03b2f5c859ac None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:54.399336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccd5f0-bc49-4ddc-b150-cb00da7b73d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:54.399968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccd5f0-bc49-4ddc-b150-cb00da7b73d2 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:54.400250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccd5f0-bc49-4ddc-b150-cb00da7b73d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:54.400531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccd5f0-bc49-4ddc-b150-cb00da7b73d2 None admin] PATH_INFO: `/v3/projects/4d7df929a74a4354a080ad68279e6d4b` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:54.401698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbccd5f0-bc49-4ddc-b150-cb00da7b73d2 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7df929a74a4354a080ad68279e6d4b)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:54.403111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbccd5f0-bc49-4ddc-b150-cb00da7b73d2 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:54.404792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 564/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:54 2022] GET /identity/v3/projects/4d7df929a74a4354a080ad68279e6d4b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:54.410803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:54.511559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53c3e8-5a02-4dda-b15f-c01f11a4b4fd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:54.537691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:54.545527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07d435-9c44-4eb9-ba24-6233f9f21357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:54.546187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d435-9c44-4eb9-ba24-6233f9f21357 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:54.546471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d435-9c44-4eb9-ba24-6233f9f21357 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:54.546750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d435-9c44-4eb9-ba24-6233f9f21357 None admin] PATH_INFO: `/v3/projects/4d7df929a74a4354a080ad68279e6d4b/users/73a448832c8e459495eccd60bac62759/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:54.549440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07d435-9c44-4eb9-ba24-6233f9f21357 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d7df929a74a4354a080ad68279e6d4b, user_id=73a448832c8e459495eccd60bac62759, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:54.550872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07d435-9c44-4eb9-ba24-6233f9f21357 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:54.564977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 589/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:14:54 2022] PUT /identity/v3/projects/4d7df929a74a4354a080ad68279e6d4b/users/73a448832c8e459495eccd60bac62759/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:14:56.538079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dace7-2ce9-4675-85d3-6bac61bbd6dc None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:56.538636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dace7-2ce9-4675-85d3-6bac61bbd6dc None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:56.538636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dace7-2ce9-4675-85d3-6bac61bbd6dc None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:56.539964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 565/1154] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:56.545200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ca289-d0a8-4d5b-b096-31c41eb460a3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:56.545615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ca289-d0a8-4d5b-b096-31c41eb460a3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:56.545753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ca289-d0a8-4d5b-b096-31c41eb460a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:56.564456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d09ca289-d0a8-4d5b-b096-31c41eb460a3 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:56.565651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d09ca289-d0a8-4d5b-b096-31c41eb460a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:56.629559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 590/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:56.651696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dace7-2ce9-4675-85d3-6bac61bbd6dc None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:56.698702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:56.751853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dace7-2ce9-4675-85d3-6bac61bbd6dc None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:56.784820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:56.785457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:56.785746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:56.786021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:56.795785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:14:56.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:14:56.802367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:14:56.803234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 566/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:14:56.808768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ca289-d0a8-4d5b-b096-31c41eb460a3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:56.910656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ca289-d0a8-4d5b-b096-31c41eb460a3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:56.928290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:56.945424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:56.946039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:56.946321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:56.946596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:56.947292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:56.947595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:56.949000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:56.957087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 591/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:56.962816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:57.062274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9185ee-1efb-4bed-bcee-6dd8c838a45b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:57.069849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:57.097509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:57.098137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:57.098424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:57.098704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] PATH_INFO: `/v3/users/73a448832c8e459495eccd60bac62759` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:57.099852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] RBAC: Authorizing `identity:get_user(user_id=73a448832c8e459495eccd60bac62759)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:57.101295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:57.102886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 567/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:14:56 2022] GET /identity/v3/users/73a448832c8e459495eccd60bac62759 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:57.108701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:57.205349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359cba-2c69-44ab-a4b3-bb551cd34010 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:57.237952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:57.238553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:57.238832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:57.239104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:57.284456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:57.303533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:57.304996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:57.399580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 592/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:14:57.405460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:57.504818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bf7e3-0b5c-4956-bae1-4cc901f2942e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:57.537341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570fe9e-d8ae-40b6-a1a1-342444500f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:57.537974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570fe9e-d8ae-40b6-a1a1-342444500f91 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:57.538258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570fe9e-d8ae-40b6-a1a1-342444500f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:57.538816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570fe9e-d8ae-40b6-a1a1-342444500f91 None admin] PATH_INFO: `/v3/projects/4d7df929a74a4354a080ad68279e6d4b` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:57.539655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570fe9e-d8ae-40b6-a1a1-342444500f91 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7df929a74a4354a080ad68279e6d4b)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:57.541109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570fe9e-d8ae-40b6-a1a1-342444500f91 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:57.542750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 568/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:14:57 2022] GET /identity/v3/projects/4d7df929a74a4354a080ad68279e6d4b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:57.549480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:57.628924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:57.648628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da33f-3f03-401e-8fc0-258ca7b23af3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:57.681070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921fc24-2ad2-4f6b-b756-a368c969e403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:57.681614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921fc24-2ad2-4f6b-b756-a368c969e403 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:57.681894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921fc24-2ad2-4f6b-b756-a368c969e403 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:57.682169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921fc24-2ad2-4f6b-b756-a368c969e403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:57.682957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921fc24-2ad2-4f6b-b756-a368c969e403 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=4d7df929a74a4354a080ad68279e6d4b, user.id=73a448832c8e459495eccd60bac62759) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:14:57.683264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921fc24-2ad2-4f6b-b756-a368c969e403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:14:57.684743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921fc24-2ad2-4f6b-b756-a368c969e403 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:14:57.691037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 593/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:14:57 2022] GET /identity/v3/role_assignments?user.id=73a448832c8e459495eccd60bac62759&scope.project.id=4d7df929a74a4354a080ad68279e6d4b&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:14:59.716977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa8c6d-d3b1-4e65-8e7f-33db5fc56e09 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:59.717514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa8c6d-d3b1-4e65-8e7f-33db5fc56e09 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:59.717514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa8c6d-d3b1-4e65-8e7f-33db5fc56e09 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:59.718897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 569/1162] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:14:59.724823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796498a-9c61-48e9-a0fe-36f15c7e2002 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:59.725456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796498a-9c61-48e9-a0fe-36f15c7e2002 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:59.725456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796498a-9c61-48e9-a0fe-36f15c7e2002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:14:59.745141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1796498a-9c61-48e9-a0fe-36f15c7e2002 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:14:59.747219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1796498a-9c61-48e9-a0fe-36f15c7e2002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:14:59.814549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 594/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:14:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:14:59.837349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa8c6d-d3b1-4e65-8e7f-33db5fc56e09 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:14:59.880945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:14:59.937543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa8c6d-d3b1-4e65-8e7f-33db5fc56e09 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:14:59.971464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:14:59.972013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:14:59.972298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:14:59.972714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:00.038805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:00.040249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:00.080982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:00.136603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 570/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:14:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:15:00.142600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796498a-9c61-48e9-a0fe-36f15c7e2002 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:00.213864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:00.239229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796498a-9c61-48e9-a0fe-36f15c7e2002 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:00.272920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97604c-c67b-4cff-8d17-4e0804bcbd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:00.273554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97604c-c67b-4cff-8d17-4e0804bcbd56 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:00.273837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97604c-c67b-4cff-8d17-4e0804bcbd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:00.274113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97604c-c67b-4cff-8d17-4e0804bcbd56 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:00.280552 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97604c-c67b-4cff-8d17-4e0804bcbd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:00.282347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97604c-c67b-4cff-8d17-4e0804bcbd56 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5a97604c-c67b-4cff-8d17-4e0804bcbd56 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:15:00.294370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:15:00.295600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:15:00.296676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 595/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:15:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:15:00.301990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:00.404870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562756a-1a5d-4474-8587-f8ba5fa5e0eb None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:00.437611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05839fd6-ef4e-4a47-999a-d05263e40ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:00.438941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05839fd6-ef4e-4a47-999a-d05263e40ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:00.439262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05839fd6-ef4e-4a47-999a-d05263e40ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:00.439532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05839fd6-ef4e-4a47-999a-d05263e40ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:00.440213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05839fd6-ef4e-4a47-999a-d05263e40ba5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:15:00.440516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05839fd6-ef4e-4a47-999a-d05263e40ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:00.441889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05839fd6-ef4e-4a47-999a-d05263e40ba5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:00.448807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 571/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 29 06:15:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:02.419352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae823f-2235-4b53-8326-a3e8c3d2e957 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:02.419920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae823f-2235-4b53-8326-a3e8c3d2e957 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:02.419920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae823f-2235-4b53-8326-a3e8c3d2e957 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:02.421361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 596/1167] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:15:02.428204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c49f0d-be50-4f9a-b62d-669710decf23 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:02.428204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c49f0d-be50-4f9a-b62d-669710decf23 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:02.428698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c49f0d-be50-4f9a-b62d-669710decf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:02.441267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:02.446551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f8c49f0d-be50-4f9a-b62d-669710decf23 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:15:02.447754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f8c49f0d-be50-4f9a-b62d-669710decf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:15:02.513098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 572/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:15:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:15:02.535800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae823f-2235-4b53-8326-a3e8c3d2e957 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:02.633677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae823f-2235-4b53-8326-a3e8c3d2e957 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:02.666882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284b2f0-9a47-4b09-9723-f7d243c3bd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:02.667491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284b2f0-9a47-4b09-9723-f7d243c3bd3c None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:02.667772 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284b2f0-9a47-4b09-9723-f7d243c3bd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:02.668046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284b2f0-9a47-4b09-9723-f7d243c3bd3c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:02.668775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284b2f0-9a47-4b09-9723-f7d243c3bd3c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:02.670191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284b2f0-9a47-4b09-9723-f7d243c3bd3c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:02.674071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:02.686206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 597/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Fri Jul 29 06:15:02 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:15:04.702900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14c5ff-b9fe-4ace-ba64-f32059ffc094 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:04.703438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14c5ff-b9fe-4ace-ba64-f32059ffc094 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:04.704075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14c5ff-b9fe-4ace-ba64-f32059ffc094 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:04.705871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 573/1170] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:15:04.710978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a191b-4bd4-402d-bcb9-7de6fe6186a7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:04.711358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a191b-4bd4-402d-bcb9-7de6fe6186a7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:04.711542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a191b-4bd4-402d-bcb9-7de6fe6186a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:04.730356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-861a191b-4bd4-402d-bcb9-7de6fe6186a7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:15:04.731699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-861a191b-4bd4-402d-bcb9-7de6fe6186a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:15:04.797964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 598/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:15:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:15:04.820596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14c5ff-b9fe-4ace-ba64-f32059ffc094 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:04.919040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14c5ff-b9fe-4ace-ba64-f32059ffc094 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:04.951409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dda3ae-733b-48ad-9fa7-e9af193e0ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:04.952052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda3ae-733b-48ad-9fa7-e9af193e0ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:04.952337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda3ae-733b-48ad-9fa7-e9af193e0ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:04.952638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda3ae-733b-48ad-9fa7-e9af193e0ce9 None admin] PATH_INFO: `/v3/domains/728a598e24d34629824c37b5638afb81` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:04.956230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:04.964374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dda3ae-733b-48ad-9fa7-e9af193e0ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=728a598e24d34629824c37b5638afb81)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:04.965831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dda3ae-733b-48ad-9fa7-e9af193e0ce9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:04.967442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 574/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Fri Jul 29 06:15:04 2022] GET /identity/v3/domains/728a598e24d34629824c37b5638afb81 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:04.973945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a191b-4bd4-402d-bcb9-7de6fe6186a7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:05.074398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a191b-4bd4-402d-bcb9-7de6fe6186a7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:05.098004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:05.109620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72786ec8-6972-4947-9b1f-e0bdca79cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:05.110587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786ec8-6972-4947-9b1f-e0bdca79cc81 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:05.110876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786ec8-6972-4947-9b1f-e0bdca79cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:05.111156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786ec8-6972-4947-9b1f-e0bdca79cc81 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:05.111830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72786ec8-6972-4947-9b1f-e0bdca79cc81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:05.113285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72786ec8-6972-4947-9b1f-e0bdca79cc81 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:05.138977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 599/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:15:04 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:15:07.158827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6573c8-3e11-4006-89b8-93183d66bc0a None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:07.159363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6573c8-3e11-4006-89b8-93183d66bc0a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:07.159363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6573c8-3e11-4006-89b8-93183d66bc0a None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:07.160736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 575/1174] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:15:07.165810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10399602-d327-4e14-a34f-45cba84812fa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:07.166220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10399602-d327-4e14-a34f-45cba84812fa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:07.166376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10399602-d327-4e14-a34f-45cba84812fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:07.184103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-10399602-d327-4e14-a34f-45cba84812fa None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:15:07.185313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-10399602-d327-4e14-a34f-45cba84812fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:15:07.249612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 600/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:15:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:15:07.274216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6573c8-3e11-4006-89b8-93183d66bc0a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:07.373948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6573c8-3e11-4006-89b8-93183d66bc0a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:07.406911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4865d-46ac-481f-a63a-7c534cf32690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:07.407617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4865d-46ac-481f-a63a-7c534cf32690 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:07.407900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4865d-46ac-481f-a63a-7c534cf32690 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:07.408177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4865d-46ac-481f-a63a-7c534cf32690 None admin] PATH_INFO: `/v3/domains/728a598e24d34629824c37b5638afb81` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:07.409383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4865d-46ac-481f-a63a-7c534cf32690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=728a598e24d34629824c37b5638afb81)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:07.410778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4865d-46ac-481f-a63a-7c534cf32690 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:07.412341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 576/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Fri Jul 29 06:15:07 2022] GET /identity/v3/domains/728a598e24d34629824c37b5638afb81 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:07.418115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10399602-d327-4e14-a34f-45cba84812fa None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:07.515297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10399602-d327-4e14-a34f-45cba84812fa None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:07.523088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:07.551282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2820699-eeef-481d-aa51-2a35c2f12c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:07.551899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820699-eeef-481d-aa51-2a35c2f12c49 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:07.552186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820699-eeef-481d-aa51-2a35c2f12c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:07.552467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820699-eeef-481d-aa51-2a35c2f12c49 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:07.553183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2820699-eeef-481d-aa51-2a35c2f12c49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:07.554612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2820699-eeef-481d-aa51-2a35c2f12c49 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:07.576914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 601/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:15:07 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:15:09.646708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9bd1-bde2-40d4-9941-b402822d67eb None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:09.647368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9bd1-bde2-40d4-9941-b402822d67eb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:09.647368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9bd1-bde2-40d4-9941-b402822d67eb None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:09.648632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 577/1178] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:15:09.653858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37113526-9e22-499c-bdd7-f02fe3294332 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:09.654257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37113526-9e22-499c-bdd7-f02fe3294332 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:09.654412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37113526-9e22-499c-bdd7-f02fe3294332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:09.673217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-37113526-9e22-499c-bdd7-f02fe3294332 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:15:09.674388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-37113526-9e22-499c-bdd7-f02fe3294332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:15:09.742615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 602/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:15:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:15:09.765107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f9bd1-bde2-40d4-9941-b402822d67eb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:09.828603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:09.868645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f9bd1-bde2-40d4-9941-b402822d67eb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:09.902402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:09.902964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:09.903242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:09.903514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:09.913811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:15:09.918612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:15:09.919775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:15:09.920870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 578/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:15:09.925779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37113526-9e22-499c-bdd7-f02fe3294332 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:10.014250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:10.024761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37113526-9e22-499c-bdd7-f02fe3294332 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:10.058168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:10.058796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:10.059084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:10.059364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:10.060078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:15:10.060391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:10.061809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:10.069872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 603/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:10.075602 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:10.175599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f616f79-6625-48d1-bdd9-4babf1ca6769 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:10.198388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:10.210010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:10.210648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:10.210929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:10.211203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] PATH_INFO: `/v3/users/9cfbdd9cb67d473592328058de6acc90` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:10.224882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=9cfbdd9cb67d473592328058de6acc90)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:10.226323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:10.227920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 579/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:15:10 2022] GET /identity/v3/users/9cfbdd9cb67d473592328058de6acc90 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:10.233648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:10.332039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e05ec-2849-4ff0-808b-e2d9d90abd8b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:10.364696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:10.365306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:10.365590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:10.365864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:10.389377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:10.432915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:10.434325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:10.530311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 604/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:15:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:15:10.536235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:10.630891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a362f-c71f-4401-8200-ddc65fdde0b0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:10.663648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1a7cf-24c8-49ab-b11d-a335184f5927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:10.664293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1a7cf-24c8-49ab-b11d-a335184f5927 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:10.664665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1a7cf-24c8-49ab-b11d-a335184f5927 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:10.664951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1a7cf-24c8-49ab-b11d-a335184f5927 None admin] PATH_INFO: `/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:10.666093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1a7cf-24c8-49ab-b11d-a335184f5927 None admin] RBAC: Authorizing `identity:get_project(project_id=4cfd13b1913f4eb7a220186a5365e2b6)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:10.667501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1a7cf-24c8-49ab-b11d-a335184f5927 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:10.669167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 580/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:15:10 2022] GET /identity/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:10.675710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:10.728920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:10.773236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39603d-350a-4b73-816f-ec0b4e5b3af0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:10.806160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc67157-b178-4a4b-a02e-d4caee4a3796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:10.806776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc67157-b178-4a4b-a02e-d4caee4a3796 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:10.807059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc67157-b178-4a4b-a02e-d4caee4a3796 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:10.807338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc67157-b178-4a4b-a02e-d4caee4a3796 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:10.808111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc67157-b178-4a4b-a02e-d4caee4a3796 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=4cfd13b1913f4eb7a220186a5365e2b6, user.id=9cfbdd9cb67d473592328058de6acc90) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:15:10.808425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc67157-b178-4a4b-a02e-d4caee4a3796 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:10.809874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc67157-b178-4a4b-a02e-d4caee4a3796 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:10.815795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 605/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:15:10 2022] GET /identity/v3/role_assignments?user.id=9cfbdd9cb67d473592328058de6acc90&scope.project.id=4cfd13b1913f4eb7a220186a5365e2b6&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:12.773936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d1de6-320c-4fd5-b0ff-614073625801 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:12.774374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d1de6-320c-4fd5-b0ff-614073625801 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:12.774485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d1de6-320c-4fd5-b0ff-614073625801 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:12.775814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 581/1186] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:15:12.780867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885399-ee34-4e7a-bb03-26fade587a2d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:12.781787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885399-ee34-4e7a-bb03-26fade587a2d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:12.782007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885399-ee34-4e7a-bb03-26fade587a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:12.800454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-93885399-ee34-4e7a-bb03-26fade587a2d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:15:12.801670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-93885399-ee34-4e7a-bb03-26fade587a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:15:12.866973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 606/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:15:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:15:12.888878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d1de6-320c-4fd5-b0ff-614073625801 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:12.938613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:12.988764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d1de6-320c-4fd5-b0ff-614073625801 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:13.022205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:13.022844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:13.023123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:13.023396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:13.033884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:15:13.038708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:15:13.039799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:15:13.040817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 582/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:15:13.045815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93885399-ee34-4e7a-bb03-26fade587a2d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:13.094619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:13.144618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93885399-ee34-4e7a-bb03-26fade587a2d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:13.177610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:13.178215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:13.178500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:13.178785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:13.179487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:15:13.179795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:13.181195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:13.189355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 607/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:15:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:13.195001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:13.291557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82971f-b965-4efb-8a4a-d166e16b85f9 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:13.302258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:13.325283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:13.325907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:13.326187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:13.326457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] PATH_INFO: `/v3/users/9cfbdd9cb67d473592328058de6acc90` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:13.327585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] RBAC: Authorizing `identity:get_user(user_id=9cfbdd9cb67d473592328058de6acc90)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:13.329570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:13.331385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 583/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:15:13 2022] GET /identity/v3/users/9cfbdd9cb67d473592328058de6acc90 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:13.337260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:13.434579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445aedf-6005-4c41-82db-ddd062a99f91 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:13.445051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:13.468194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3391fb1-6603-49ac-9fa6-363b42ac369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:13.468993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391fb1-6603-49ac-9fa6-363b42ac369b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:13.469507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391fb1-6603-49ac-9fa6-363b42ac369b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:13.469804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391fb1-6603-49ac-9fa6-363b42ac369b None admin] PATH_INFO: `/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:13.470931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3391fb1-6603-49ac-9fa6-363b42ac369b None admin] RBAC: Authorizing `identity:get_project(project_id=4cfd13b1913f4eb7a220186a5365e2b6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:13.472341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3391fb1-6603-49ac-9fa6-363b42ac369b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:13.474007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 608/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:15:13 2022] GET /identity/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:13.479980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:13.577875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bde92e-23cc-4d56-8b8d-45e0a7c5bd0b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:13.611674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875558c5-6d05-4e4a-acc2-a5c7400902cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:13.612329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875558c5-6d05-4e4a-acc2-a5c7400902cf None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:13.612640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875558c5-6d05-4e4a-acc2-a5c7400902cf None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:13.612922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875558c5-6d05-4e4a-acc2-a5c7400902cf None admin] PATH_INFO: `/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6/users/9cfbdd9cb67d473592328058de6acc90/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:13.614832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875558c5-6d05-4e4a-acc2-a5c7400902cf None admin] RBAC: Authorizing `identity:create_grant(project_id=4cfd13b1913f4eb7a220186a5365e2b6, user_id=9cfbdd9cb67d473592328058de6acc90, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:13.616241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875558c5-6d05-4e4a-acc2-a5c7400902cf None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:13.629974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 584/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:15:13 2022] PUT /identity/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6/users/9cfbdd9cb67d473592328058de6acc90/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:15:15.585752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240e6d8-0df7-4898-97fe-bd55a447eacb None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:15.586324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240e6d8-0df7-4898-97fe-bd55a447eacb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:15.586324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240e6d8-0df7-4898-97fe-bd55a447eacb None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:15.587657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 609/1193] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:15:15.592826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220429c6-68c1-4c5f-8b72-ae3a16a4307e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:15.593238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220429c6-68c1-4c5f-8b72-ae3a16a4307e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:15.593372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220429c6-68c1-4c5f-8b72-ae3a16a4307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:15.612724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-220429c6-68c1-4c5f-8b72-ae3a16a4307e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:15:15.613956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-220429c6-68c1-4c5f-8b72-ae3a16a4307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:15:15.643195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:15.681749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 585/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:15:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:15:15.705788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240e6d8-0df7-4898-97fe-bd55a447eacb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:15.808885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240e6d8-0df7-4898-97fe-bd55a447eacb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:15.841964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:15.842543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:15.842827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:15.843103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:15.854160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:15:15.859025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:15:15.860985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:15:15.862087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 610/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:15:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:15:15.867223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220429c6-68c1-4c5f-8b72-ae3a16a4307e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:15.967753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220429c6-68c1-4c5f-8b72-ae3a16a4307e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:16.008062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:16.008719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:16.009005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:16.009284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:16.009989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:15:16.010297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:16.011656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:16.019983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 586/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:15:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:16.025770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:16.065701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:16.124676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081e5a7-bc0c-4cfb-8607-337a1b4ffe1f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:16.157801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e27c9-643d-4921-a863-f843174c812c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:16.158434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e27c9-643d-4921-a863-f843174c812c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:16.158716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e27c9-643d-4921-a863-f843174c812c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:16.158994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e27c9-643d-4921-a863-f843174c812c None admin] PATH_INFO: `/v3/users/9cfbdd9cb67d473592328058de6acc90` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:16.160147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e27c9-643d-4921-a863-f843174c812c None admin] RBAC: Authorizing `identity:get_user(user_id=9cfbdd9cb67d473592328058de6acc90)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:16.161604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e27c9-643d-4921-a863-f843174c812c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:16.163260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 611/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 29 06:15:16 2022] GET /identity/v3/users/9cfbdd9cb67d473592328058de6acc90 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:16.168931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:16.198968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:16.269261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687c687-6477-4b53-aaa2-3241c1f8dc50 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:16.301821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:16.302430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:16.302713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:16.302989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:16.367604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:16.369072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:16.397501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:16.465362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 587/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:15:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:15:16.471697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e27c9-643d-4921-a863-f843174c812c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:16.568112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e27c9-643d-4921-a863-f843174c812c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:16.574641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:16.601460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb47e1-a7bd-40c4-af48-85a9b5d76ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:16.602095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb47e1-a7bd-40c4-af48-85a9b5d76ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:16.602372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb47e1-a7bd-40c4-af48-85a9b5d76ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:16.602644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb47e1-a7bd-40c4-af48-85a9b5d76ce7 None admin] PATH_INFO: `/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:16.603768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb47e1-a7bd-40c4-af48-85a9b5d76ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=4cfd13b1913f4eb7a220186a5365e2b6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:16.605220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb47e1-a7bd-40c4-af48-85a9b5d76ce7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:16.606854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 612/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 29 06:15:16 2022] GET /identity/v3/projects/4cfd13b1913f4eb7a220186a5365e2b6 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:16.613475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:15:16.710743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e687612-7908-4b40-aa78-4cf4a0e0f85a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:15:16.739503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:16.745809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981807f5-baeb-479c-951c-2e5f4702baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:15:16.746431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981807f5-baeb-479c-951c-2e5f4702baa8 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:15:16.746711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981807f5-baeb-479c-951c-2e5f4702baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:15:16.746986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981807f5-baeb-479c-951c-2e5f4702baa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:15:16.747761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981807f5-baeb-479c-951c-2e5f4702baa8 None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=4cfd13b1913f4eb7a220186a5365e2b6, user.id=9cfbdd9cb67d473592328058de6acc90) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:15:16.748071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981807f5-baeb-479c-951c-2e5f4702baa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:15:16.749522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981807f5-baeb-479c-951c-2e5f4702baa8 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:15:16.755741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 588/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:15:16 2022] GET /identity/v3/role_assignments?user.id=9cfbdd9cb67d473592328058de6acc90&scope.project.id=4cfd13b1913f4eb7a220186a5365e2b6&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:15:26.597460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=13 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:15:26.753181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:11.709213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008ba8-3452-4391-92d3-741ebbb07bd7 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:11.709794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008ba8-3452-4391-92d3-741ebbb07bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:11.709794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008ba8-3452-4391-92d3-741ebbb07bd7 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:11.711337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 613/1201] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:11.717169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac918-3b27-48ee-8b81-ad61ccc38659 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:11.717656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac918-3b27-48ee-8b81-ad61ccc38659 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:11.717743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac918-3b27-48ee-8b81-ad61ccc38659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:11.745485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-13dac918-3b27-48ee-8b81-ad61ccc38659 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:11.746824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-13dac918-3b27-48ee-8b81-ad61ccc38659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:11.836934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 589/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:16:11.861872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7008ba8-3452-4391-92d3-741ebbb07bd7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:11.977105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7008ba8-3452-4391-92d3-741ebbb07bd7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:12.018366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:12.019017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:12.019295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:12.019566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:12.031360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:12.032959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:12.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:12.049392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:12.051009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:12.051009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:12.051009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:12.051009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:12.051009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 614/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:12.058088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac918-3b27-48ee-8b81-ad61ccc38659 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:12.183130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac918-3b27-48ee-8b81-ad61ccc38659 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:12.220669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:12.229685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617adf2-535f-4106-bf77-ff3054873d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:12.230251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617adf2-535f-4106-bf77-ff3054873d1e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:12.230532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617adf2-535f-4106-bf77-ff3054873d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:12.230808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617adf2-535f-4106-bf77-ff3054873d1e None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:12.231502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617adf2-535f-4106-bf77-ff3054873d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:12.231806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617adf2-535f-4106-bf77-ff3054873d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:12.233329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617adf2-535f-4106-bf77-ff3054873d1e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:12.252592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 590/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:16:12.255978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:12.325623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:12.391071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b35fb1-08e4-4fc6-9190-1b6a2c70c239 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:12.440742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164acc8-f405-41c1-9985-41cfafeb6945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:12.441739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164acc8-f405-41c1-9985-41cfafeb6945 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:12.442189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164acc8-f405-41c1-9985-41cfafeb6945 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:12.442538 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164acc8-f405-41c1-9985-41cfafeb6945 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:12.443521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164acc8-f405-41c1-9985-41cfafeb6945 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:12.445335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164acc8-f405-41c1-9985-41cfafeb6945 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:12.475058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 615/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 29 06:16:12 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:16:14.945332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9105-3395-45f3-a825-1388914665a9 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:14.945919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9105-3395-45f3-a825-1388914665a9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:14.945919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9105-3395-45f3-a825-1388914665a9 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:14.947473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 591/1206] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:14.954958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85931b5b-a0cc-45f3-b1ad-af249781f100 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:14.955699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85931b5b-a0cc-45f3-b1ad-af249781f100 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:14.955798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85931b5b-a0cc-45f3-b1ad-af249781f100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:14.974915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-85931b5b-a0cc-45f3-b1ad-af249781f100 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:14.976059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-85931b5b-a0cc-45f3-b1ad-af249781f100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:15.058674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 616/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:16:15.089167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce9105-3395-45f3-a825-1388914665a9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:15.217084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce9105-3395-45f3-a825-1388914665a9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:15.254697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:15.255359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:15.255661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:15.255940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:15.267893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:16:15.273233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:15.274633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:15.275479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 592/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:16:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:15.280813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85931b5b-a0cc-45f3-b1ad-af249781f100 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:15.335692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:15.402064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85931b5b-a0cc-45f3-b1ad-af249781f100 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:15.445896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:15.446694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:15.447015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:15.447386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:15.448200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:15.448565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:15.450519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:15.461881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 617/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:16:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:15.467962 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:15.525742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:15.587685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b192651-0293-45d1-ac40-6a59ffbc2ecc None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:15.635947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:15.636642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:15.636988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:15.637269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:15.651109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:15.653587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:15.674490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:15.675489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:15.676431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:15.676431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:15.676431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:15.676431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:15.676431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 593/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:15.681541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:15.800007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:15.803381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2104ab-8459-428e-a5fd-d70a6c672ea1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:15.847662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:15.848307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:15.848711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:15.848921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:15.849648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:15.849933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:15.851291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:15.865247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 618/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:16:15.871145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:16.002545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba761-b849-4dff-9c6e-1ac273e2ca1b None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:16.004478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:16.043123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:16.043839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:16.044138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:16.045162 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:16.053238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:16.054697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:16:16.068599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:16.069685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:16.070653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 594/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:16.076016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:16.195362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b38527-d567-47ce-92b7-099fdd4975c5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:16.239422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:16.240714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:16.240714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:16.240932 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:16.241747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:16.242095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:16.243646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:16.264849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 619/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 29 06:16:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:16.270522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:16.395169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27c14c-5b98-4c5f-881b-cd6e80bd1546 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:16.436990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:16.438270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:16.438270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:16.438270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:16.480226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:16.531034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:16.532854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:16.650333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 595/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:16:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:16:16.655993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:16.678253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:16.793494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76f48-0f6b-4283-a571-62de1c73eae5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:16.847601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:16.848497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:16.849862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:16.849862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:16.865195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:16.867890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:16.892449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:16.893422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:16.894665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:16.894665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:16.894665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:16.894665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:16.894665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 620/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 06:16:16.902188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:17.001267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:17.060309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adafafa-402c-431d-a8dc-f6c0f7b04f21 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:17.111929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:17.112723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:17.113084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:17.113434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:17.114211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:17.114592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:17.116029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:17.130323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 596/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:17.136861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:17.214546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:17.251805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af1cf9-024e-4082-9651-059684667e22 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:17.289798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:17.290651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:17.291067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:17.291439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:17.299311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:17.300908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:16:17.315191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:17.316339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:17.317490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:17.325411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 621/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:17.325588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:17.447177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67287a2e-aced-41b2-9bde-5b681c7102e7 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:17.476026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:17.490458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e73b6a-89e1-47af-9d55-779f64c13027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:17.491170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73b6a-89e1-47af-9d55-779f64c13027 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:17.491524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73b6a-89e1-47af-9d55-779f64c13027 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:17.491875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e73b6a-89e1-47af-9d55-779f64c13027 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:17.492720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e73b6a-89e1-47af-9d55-779f64c13027 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:17.493098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e73b6a-89e1-47af-9d55-779f64c13027 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:17.494806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e73b6a-89e1-47af-9d55-779f64c13027 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:17.508691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 597/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:16:17.514947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:17.632207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e433f-13d8-4449-bf2a-23e69a2c99b9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:17.660713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:17.667241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15853c02-c130-4e61-b8b5-5884475e7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:17.668000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853c02-c130-4e61-b8b5-5884475e7027 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:17.668358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853c02-c130-4e61-b8b5-5884475e7027 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:17.668755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853c02-c130-4e61-b8b5-5884475e7027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:17.670284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15853c02-c130-4e61-b8b5-5884475e7027 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=b4986a5d3e6c4adf832c1b1ec8dd768a) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:17.670630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15853c02-c130-4e61-b8b5-5884475e7027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:17.672043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15853c02-c130-4e61-b8b5-5884475e7027 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:17.688564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 622/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:16:17 2022] GET /identity/v3/role_assignments?user.id=b4986a5d3e6c4adf832c1b1ec8dd768a&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:19.807377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb72f63-701e-4f4a-a6e5-db44e996c423 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:19.807936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb72f63-701e-4f4a-a6e5-db44e996c423 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:19.807936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb72f63-701e-4f4a-a6e5-db44e996c423 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:19.809310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 598/1220] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:19.815057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3a60b-0718-437b-b353-3de16009c2c5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:19.815643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3a60b-0718-437b-b353-3de16009c2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:19.815793 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3a60b-0718-437b-b353-3de16009c2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:19.836627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8ee3a60b-0718-437b-b353-3de16009c2c5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:19.837811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8ee3a60b-0718-437b-b353-3de16009c2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:19.912466 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 623/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:19.935458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb72f63-701e-4f4a-a6e5-db44e996c423 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:20.052180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb72f63-701e-4f4a-a6e5-db44e996c423 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:20.092088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804d524-0562-4468-8446-bc120e57292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:20.092795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804d524-0562-4468-8446-bc120e57292f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:20.093079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804d524-0562-4468-8446-bc120e57292f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:20.093355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804d524-0562-4468-8446-bc120e57292f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:20.104629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804d524-0562-4468-8446-bc120e57292f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-d804d524-0562-4468-8446-bc120e57292f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:16:20.109823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:20.110952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:20.112080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 599/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:20.117133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3a60b-0718-437b-b353-3de16009c2c5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:20.226041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3a60b-0718-437b-b353-3de16009c2c5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:20.251339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:20.264252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:20.264899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:20.265208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:20.265447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:20.266218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:20.266590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:20.267989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:20.276332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 624/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:16:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:20.282201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804d524-0562-4468-8446-bc120e57292f None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:20.329321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:20.388706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804d524-0562-4468-8446-bc120e57292f None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:20.424660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:20.425344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:20.425626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:20.425921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:20.437902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:20.439514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:20.457539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:20.459020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:20.460134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:20.460134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:20.460134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:20.460134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:20.464064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 600/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:20.465101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:20.574181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9a27f-bb99-43b2-afa3-e965839b032e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:20.608515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:20.609147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:20.609435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:20.609738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:20.610396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:20.610702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:20.612070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:20.623872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 625/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:20.629983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:20.731832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:20.739664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f40f71-6115-4075-812e-3803f8fca140 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:20.777620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:20.778270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:20.782582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:20.782878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:20.790033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:20.791461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:16:20.803743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:20.804752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:20.805692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 601/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:20.813078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:20.864964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:20.927573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d49a7d-6622-4324-b53c-b92fe0202cb7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:20.962973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:20.963632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:20.963994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:20.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:20.965271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:20.965585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:20.967342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:20.989045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 626/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 29 06:16:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:20.995221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:21.111917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983877-8d92-4b08-baa8-cb9e09965a77 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:21.151450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:21.152030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:21.152346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:21.152700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:21.164153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:21.165622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:21.181523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:21.182613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:21.183805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:21.183805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:21.183805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:21.183805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:21.183805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 602/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:21.189630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:21.300656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e1c7d-7fa7-4401-af39-ede611a763e5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:21.303475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:21.340248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:21.340947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:21.341804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:21.342076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:21.342905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:21.343178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:21.344578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:21.357092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 627/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:21.363014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:21.373189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:21.476993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75b406-20e4-4b3b-aacc-9ba98b5cfba3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:21.513370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:21.514393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:21.514692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:21.515006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:21.522310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:21.524134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:16:21.536030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:21.537142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:21.538513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 603/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:21.543143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:21.660944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809404a-21ac-4bc6-8c89-26e56e1b0c11 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:21.703356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e34e55-684c-4a93-bb98-6fbcf7c03045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:21.704143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e34e55-684c-4a93-bb98-6fbcf7c03045 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:21.704434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e34e55-684c-4a93-bb98-6fbcf7c03045 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:21.704819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e34e55-684c-4a93-bb98-6fbcf7c03045 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:21.705990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e34e55-684c-4a93-bb98-6fbcf7c03045 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:21.706343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e34e55-684c-4a93-bb98-6fbcf7c03045 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:21.708284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e34e55-684c-4a93-bb98-6fbcf7c03045 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:21.723238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 628/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:21.729945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:21.801982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:21.840358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228257f-7507-4466-92f4-3ff5e89fc506 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:21.875751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2659df-470c-447c-aaaf-0d3672cf209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:21.876422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2659df-470c-447c-aaaf-0d3672cf209e None admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:21.876767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2659df-470c-447c-aaaf-0d3672cf209e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:21.877045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2659df-470c-447c-aaaf-0d3672cf209e None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/b4986a5d3e6c4adf832c1b1ec8dd768a/roles/b8e60301309b44f4acfea6244f79788b` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:21.891387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2659df-470c-447c-aaaf-0d3672cf209e None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=b4986a5d3e6c4adf832c1b1ec8dd768a, role_id=b8e60301309b44f4acfea6244f79788b)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:21.893036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2659df-470c-447c-aaaf-0d3672cf209e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:21.906506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 604/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:16:21 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/b4986a5d3e6c4adf832c1b1ec8dd768a/roles/b8e60301309b44f4acfea6244f79788b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:16:24.180235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62962-fac1-4df0-93aa-effa725499f3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:24.180828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62962-fac1-4df0-93aa-effa725499f3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:24.180828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62962-fac1-4df0-93aa-effa725499f3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:24.182170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 629/1233] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:24.187615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2b75d-6bae-4985-b15e-e3583f253831 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:24.188050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2b75d-6bae-4985-b15e-e3583f253831 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:24.188144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2b75d-6bae-4985-b15e-e3583f253831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:24.207763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f0a2b75d-6bae-4985-b15e-e3583f253831 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:24.209001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f0a2b75d-6bae-4985-b15e-e3583f253831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:24.282509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 605/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:24.305384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62962-fac1-4df0-93aa-effa725499f3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:24.333181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:24.415302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62962-fac1-4df0-93aa-effa725499f3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:24.455887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:24.456461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:24.456778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:24.457060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:24.468531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:16:24.473454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:24.474418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:24.475294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 630/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 29 06:16:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:24.482500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2b75d-6bae-4985-b15e-e3583f253831 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:24.532413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:24.591026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2b75d-6bae-4985-b15e-e3583f253831 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:24.629122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:24.629755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:24.630046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:24.630347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:24.631063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] RBAC: Adding query filter params (name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:24.631373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:24.632779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:24.641710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 606/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 29 06:16:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:16:24.647283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:24.745251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:24.764160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076df3fa-ecc3-4577-855b-04b2c74a7331 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:24.802190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:24.802910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:24.803346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:24.803715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:24.817548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:24.819058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:24.835487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:24.836442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:24.837385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:24.837385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:24.837385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:24.837385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:24.837385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 631/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:24.842956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:24.957635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547fb1-8132-42cf-9b38-9e5b8afb0917 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:24.967101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:24.993734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:24.994274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:24.994584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:24.994832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:24.995553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:24.995863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:24.998020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:25.014918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 607/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:25.016389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:25.118835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f5304-9dad-4617-ad48-ca9bbed32051 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:25.155977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:25.156663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:25.156949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:25.157227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:25.159946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:25.165725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:25.167342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:16:25.179508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:25.186638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:25.187735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 632/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:25.188332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:25.300961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252f97-b545-438d-b007-ee6aefc9d96d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:25.336346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:25.337002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:25.337289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:25.337567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:25.338358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:25.338669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:25.340238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:25.361478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 608/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 29 06:16:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:25.367399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:25.491511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb001b2-e67b-448d-8d49-018dc551b3fd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:25.530667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55134386-8652-4949-82ff-899630df0228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:25.531330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55134386-8652-4949-82ff-899630df0228 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:25.531664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55134386-8652-4949-82ff-899630df0228 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:25.532002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55134386-8652-4949-82ff-899630df0228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:25.600318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:25.608209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55134386-8652-4949-82ff-899630df0228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:25.609699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55134386-8652-4949-82ff-899630df0228 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:25.726815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 633/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:16:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:16:25.728402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:25.760495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:25.845714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951cadd-90f4-4250-9870-12656fb91e16 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:25.885898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:25.886537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:25.886819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:25.887098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:25.898563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:25.900257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:25.915873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:25.922272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:25.923336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:25.923336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:25.923336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:25.923336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:25.923336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 609/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:25.923619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55134386-8652-4949-82ff-899630df0228 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:26.031684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:26.038172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55134386-8652-4949-82ff-899630df0228 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:26.079064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:26.079691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:26.080269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:26.080730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:26.081424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:26.081764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:26.083568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:26.097343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 634/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:26.103955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:26.194018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:26.219419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9659-e152-4ac2-ae46-0761268c1953 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:26.258301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:26.258994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:26.259277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:26.259559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:26.267274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:26.268749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:16:26.281448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:26.282478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:26.283364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 610/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:26.288991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:26.406618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6736053-57ec-4255-94b7-1030f46785f5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:26.446411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c68668-762c-49f4-b00e-17ae704c98f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:26.447028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c68668-762c-49f4-b00e-17ae704c98f5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:26.447340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c68668-762c-49f4-b00e-17ae704c98f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:26.447611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c68668-762c-49f4-b00e-17ae704c98f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:26.448352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c68668-762c-49f4-b00e-17ae704c98f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:26.448729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c68668-762c-49f4-b00e-17ae704c98f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:26.452124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c68668-762c-49f4-b00e-17ae704c98f5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:26.466956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 635/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:26.473672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:26.584128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0892-cfd6-410b-84d6-3b4bc8526a21 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:26.620620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1bcc2-88f8-4ff0-b690-2dba70999891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:26.621309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1bcc2-88f8-4ff0-b690-2dba70999891 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:26.621592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1bcc2-88f8-4ff0-b690-2dba70999891 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:26.621870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1bcc2-88f8-4ff0-b690-2dba70999891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:26.623278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1bcc2-88f8-4ff0-b690-2dba70999891 None admin] RBAC: Adding query filter params (role.id=b8e60301309b44f4acfea6244f79788b, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=b4986a5d3e6c4adf832c1b1ec8dd768a) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:26.623586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1bcc2-88f8-4ff0-b690-2dba70999891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:26.625501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1bcc2-88f8-4ff0-b690-2dba70999891 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:26.632107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 611/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:16:26 2022] GET /identity/v3/role_assignments?user.id=b4986a5d3e6c4adf832c1b1ec8dd768a&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=b8e60301309b44f4acfea6244f79788b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:28.861803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371dee-1cd5-469f-86fe-fc0a8a02728d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:28.862354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371dee-1cd5-469f-86fe-fc0a8a02728d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:28.862354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371dee-1cd5-469f-86fe-fc0a8a02728d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:28.863715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 636/1247] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:28.868997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf6276-182e-42c0-8ad2-61ca3b6a82ea None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:28.869423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf6276-182e-42c0-8ad2-61ca3b6a82ea None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:28.869565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf6276-182e-42c0-8ad2-61ca3b6a82ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:28.876103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:28.889863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-70cf6276-182e-42c0-8ad2-61ca3b6a82ea None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:28.891065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-70cf6276-182e-42c0-8ad2-61ca3b6a82ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:28.967742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 612/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:28.994867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371dee-1cd5-469f-86fe-fc0a8a02728d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:29.014817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:29.112609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371dee-1cd5-469f-86fe-fc0a8a02728d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:29.154705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:29.155470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:29.155753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:29.156028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:29.169600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:16:29.176790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:29.177942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:29.178822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 637/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:29.185184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf6276-182e-42c0-8ad2-61ca3b6a82ea None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:29.297891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf6276-182e-42c0-8ad2-61ca3b6a82ea None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:29.334705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:29.341905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:29.342758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:29.343165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:29.343481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:29.344343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:29.344686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:29.346128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:29.357793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 613/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:16:29.363540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:29.459961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:29.490110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f67c1-5fb7-45ae-a1c6-5f4da6797a17 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:29.534234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:29.534890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:29.535165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:29.535442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:29.547104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:29.548662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:29.563821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:29.565105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:29.566406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:29.566406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:29.566406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:29.566406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:29.569881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 638/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:29.571237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:29.689999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f734c-d927-4f67-86e8-d31479c09b37 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:29.735137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:29.735768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:29.736045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:29.736314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:29.737237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:29.737669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:29.739499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:29.754337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 614/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:16:29.760247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:29.873402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb59d3-6e51-4e66-98a1-bcdc07dbc946 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:29.905613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:29.914254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:29.915062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:29.915347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:29.915697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:29.926493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:29.928043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:16:29.943794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:29.945019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:29.945978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 639/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:29.951498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:30.018745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:30.067439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0aa1d-004f-49d7-9c1a-ae4326eef8a0 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:30.105337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:30.106019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:30.106859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:30.107326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:30.108329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:30.108730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:30.110403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:30.134682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 615/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 29 06:16:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:30.135677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:30.249725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff215e5e-c8fc-4e55-8e8e-9038f1772614 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:30.290593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:30.291224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:30.291583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:30.291915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:30.360710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:30.374448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:30.375900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:30.496612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 640/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:16:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:16:30.500714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:30.604739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709acc37-bf5d-4da7-8305-3476680d9c0a None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:30.618139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:30.645986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:30.646586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:30.646864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:30.647137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:30.657743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:30.659171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:30.675532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:30.682381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:30.683307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:30.683307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:30.683307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:30.683307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:30.683307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 616/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:30.683616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:30.787503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:30.800667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70cc4-492a-42eb-b52d-3cf1a717734f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:30.838537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:30.839156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:30.839433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:30.839740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:30.840417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:30.841277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:30.842664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:30.854296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 641/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:30.860580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:30.971016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2988022-d093-43d1-baf6-7da393d3773c None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:31.004220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:31.004905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:31.005183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:31.005454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:31.012005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:31.014003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:16:31.024945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:31.026090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:31.027126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 617/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:31.032666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:31.146664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820276f9-4e95-4613-98ef-b97c77103da0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:31.183704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef2614-976a-4e6d-b9df-b14af5710444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:31.184478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef2614-976a-4e6d-b9df-b14af5710444 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:31.184881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef2614-976a-4e6d-b9df-b14af5710444 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:31.185047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef2614-976a-4e6d-b9df-b14af5710444 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:31.185750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef2614-976a-4e6d-b9df-b14af5710444 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:31.186061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef2614-976a-4e6d-b9df-b14af5710444 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:31.187495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef2614-976a-4e6d-b9df-b14af5710444 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:31.201471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 642/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:31.208148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:31.236688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:31.316827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722fdff-ed5f-4d8d-b17c-cb8495130f48 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:31.350682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea6960-6c73-41cb-8df0-11f7d16563dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:31.351322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6960-6c73-41cb-8df0-11f7d16563dc None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:31.351604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6960-6c73-41cb-8df0-11f7d16563dc None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:31.351876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6960-6c73-41cb-8df0-11f7d16563dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:31.352682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea6960-6c73-41cb-8df0-11f7d16563dc None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=b4986a5d3e6c4adf832c1b1ec8dd768a) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:31.352990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea6960-6c73-41cb-8df0-11f7d16563dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:31.354395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea6960-6c73-41cb-8df0-11f7d16563dc None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:31.360327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 618/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:16:31 2022] GET /identity/v3/role_assignments?user.id=b4986a5d3e6c4adf832c1b1ec8dd768a&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:33.417395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a37ce5-1b29-4163-a3a4-bc812dd88ed3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:33.418034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a37ce5-1b29-4163-a3a4-bc812dd88ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:33.418034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a37ce5-1b29-4163-a3a4-bc812dd88ed3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:33.420263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 643/1261] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:33.425325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b479b36-d4c7-46dc-b703-e62093b08bcb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:33.425704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b479b36-d4c7-46dc-b703-e62093b08bcb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:33.425875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b479b36-d4c7-46dc-b703-e62093b08bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:33.446676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8b479b36-d4c7-46dc-b703-e62093b08bcb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:33.448093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8b479b36-d4c7-46dc-b703-e62093b08bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:33.520287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 619/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:33.542664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a37ce5-1b29-4163-a3a4-bc812dd88ed3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:33.553917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:33.643405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a37ce5-1b29-4163-a3a4-bc812dd88ed3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:33.677446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:33.677999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:33.678285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:33.678563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:33.688398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:16:33.693297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:33.694397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:33.695603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 644/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:33.700363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b479b36-d4c7-46dc-b703-e62093b08bcb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:33.719842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:33.797920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b479b36-d4c7-46dc-b703-e62093b08bcb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:33.830235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:33.830855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:33.831139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:33.831415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:33.832118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:33.832451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:33.833968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:33.842275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 620/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:16:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:33.847959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:33.915550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:33.948251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b963d6b-c02e-43a2-9b07-3077edd6946d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:33.983040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:33.983668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:33.983950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:33.984224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:33.993858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:33.995290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:34.009877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:34.010887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:34.012154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:34.012154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:34.012154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:34.012154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:34.012154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 645/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:34.017670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:34.096456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:34.115236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc285fa2-47c1-4ec9-ac5e-30ec08fd48da None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:34.147684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:34.148296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:34.148607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:34.148890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:34.149569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:34.149877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:34.151225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:34.163250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 621/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:34.168928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:34.265726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd3fd0-2223-42dd-8663-a84e4913da3d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:34.284262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:34.300590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:34.301230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:34.301514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:34.301794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:34.309701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:34.311132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:16:34.322052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:34.323151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:34.324314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 646/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:34.329593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:34.425542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163035e-670f-49c5-99c0-6807e17dac27 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:34.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:34.464391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:34.464769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:34.465074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:34.466208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:34.466498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:34.468296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:34.478579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:34.487479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 622/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 29 06:16:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:16:34.493138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:34.592934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6328a6d-ed8b-4272-8001-3ff90e884427 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:34.625876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:34.626502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:34.626787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:34.627067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:34.637175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:34.638845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:34.652592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:34.653568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:34.654689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:34.654689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:34.654689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:34.654689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:34.654689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 647/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:34.659350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:34.756587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6a154-b67c-47be-a007-8f0349e665b3 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:34.789754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:34.790369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:34.790653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:34.790929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:34.791607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:34.791911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:34.793300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:34.805049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 623/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:34.811679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:34.842564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:34.910339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478a5b-acff-46c4-a590-c30cdebc5e2e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:34.943005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:34.943625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:34.943905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:34.944176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:34.950583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:34.951999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:16:34.963281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:34.964319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:34.965448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 648/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:34.970328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:35.027359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:35.069921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c26c66-8547-4292-a45b-c0e2d2d88449 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:35.102235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e91bf-c06d-4ef5-95ed-dd26cbe95896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:35.102839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e91bf-c06d-4ef5-95ed-dd26cbe95896 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:35.103121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e91bf-c06d-4ef5-95ed-dd26cbe95896 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:35.103398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e91bf-c06d-4ef5-95ed-dd26cbe95896 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:35.104131 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e91bf-c06d-4ef5-95ed-dd26cbe95896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:35.104438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e91bf-c06d-4ef5-95ed-dd26cbe95896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:35.105938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e91bf-c06d-4ef5-95ed-dd26cbe95896 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:35.118116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 624/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:35.124165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:35.211455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:35.220475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29439b3-0c29-483c-a7c3-aa102e2100a2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:35.253016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e98506-53d8-48e2-b338-805914f588f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:35.253666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e98506-53d8-48e2-b338-805914f588f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:35.253950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e98506-53d8-48e2-b338-805914f588f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:35.254225 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e98506-53d8-48e2-b338-805914f588f5 None admin] PATH_INFO: `/v3/projects/b42836f77f104b648efac765cddf6b93/users/b4986a5d3e6c4adf832c1b1ec8dd768a/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:35.256099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e98506-53d8-48e2-b338-805914f588f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b42836f77f104b648efac765cddf6b93, user_id=b4986a5d3e6c4adf832c1b1ec8dd768a, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:35.257542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e98506-53d8-48e2-b338-805914f588f5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:35.269652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 649/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 29 06:16:35 2022] PUT /identity/v3/projects/b42836f77f104b648efac765cddf6b93/users/b4986a5d3e6c4adf832c1b1ec8dd768a/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:16:37.213404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a3c43-22c0-4655-879c-13c96081296d None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:37.213939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a3c43-22c0-4655-879c-13c96081296d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:37.213939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a3c43-22c0-4655-879c-13c96081296d None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:37.215922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 625/1274] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:37.220884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080153a-ea1a-4c08-a8fc-842670cf1ea2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:37.221286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080153a-ea1a-4c08-a8fc-842670cf1ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:37.221441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080153a-ea1a-4c08-a8fc-842670cf1ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:37.239137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9080153a-ea1a-4c08-a8fc-842670cf1ea2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:37.240307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9080153a-ea1a-4c08-a8fc-842670cf1ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:37.303840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 650/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:37.327406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a3c43-22c0-4655-879c-13c96081296d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:37.426795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a3c43-22c0-4655-879c-13c96081296d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:37.445797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:37.467736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:37.468312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:37.468653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:37.468936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:37.479911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 06:16:37.485209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:37.486313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:37.487342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 626/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 29 06:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:37.492642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080153a-ea1a-4c08-a8fc-842670cf1ea2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:37.573128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:37.592882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080153a-ea1a-4c08-a8fc-842670cf1ea2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:37.632343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:37.633368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:37.633846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:37.634309 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:37.635523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:37.636074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:37.638049 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:37.646396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 651/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 29 06:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:37.652019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:37.750179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635b3fe-ed9d-4112-9828-d556aa76b104 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:37.783159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:37.783795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:37.784078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:37.784355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:37.794834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:37.796264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:37.811188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:37.812232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:37.813427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:37.813427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:37.813427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:37.813427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:37.813427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 627/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:37.818091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:37.916469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca46b6-1d2d-44a4-953c-ff3cb45a476b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:37.949790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:37.950620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:37.950901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:37.951180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:37.951859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:37.952165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:37.953575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:37.965150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 652/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:37.970845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:38.001867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:38.070033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfbc93-0d67-4dcd-b476-d0f68720ff72 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:38.102802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:38.103432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:38.103716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:38.103993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:38.111811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:38.113282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 06:16:38.125130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:38.126318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:38.127262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 628/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:38.132141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:38.135898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:38.229737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefb704-bfd8-406c-acc2-f67097559556 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:38.263299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:38.263858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:38.264143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:38.264420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:38.265230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:38.265539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:38.266992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:38.284038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 653/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 29 06:16:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:38.289591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:38.380427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:38.390894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fbbd6-e7d5-40f4-a927-9562a326c3ea None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:38.423408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:38.424024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:38.424309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:38.424630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:38.497974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:38.500499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:38.593209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:38.603316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 629/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 29 06:16:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 06:16:38.609611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:38.669618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:38.708191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e3b2-396d-4aaa-baad-01a0e7641b6c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:38.742002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:38.742644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:38.742928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:38.743205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:38.753849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:38.755274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 06:16:38.770363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:38.771407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:38.772371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 06:16:38.772371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 06:16:38.772371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 06:16:38.772371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:38.772371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 654/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:38.777124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:38.875416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b99d8-a0bb-4845-9eaf-6790ed5ee17d None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:38.910439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:38.911049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:38.911330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:38.911607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:38.912289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:38.912625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:38.913998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:38.925387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 630/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 29 06:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:38.931003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:39.031371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c5bad-5545-42aa-8320-ef99e641ffc2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:39.044837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:39.065525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:39.066153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:39.066437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:39.066713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:39.073117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:39.075117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 06:16:39.086827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:39.087862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jul 29 06:16:39.088874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 655/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 29 06:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:39.093957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:39.142457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:39.190556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797489b-aaf2-4ca8-ba37-f11ffca2731f None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:39.222068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feca074-36cf-4e71-b164-2626cba04ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:39.222665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feca074-36cf-4e71-b164-2626cba04ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:39.222945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feca074-36cf-4e71-b164-2626cba04ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:39.223220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feca074-36cf-4e71-b164-2626cba04ea5 None admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:39.223944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feca074-36cf-4e71-b164-2626cba04ea5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:39.224248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feca074-36cf-4e71-b164-2626cba04ea5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:39.225673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feca074-36cf-4e71-b164-2626cba04ea5 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:39.237577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 631/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 29 06:16:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:39.243426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:39.341458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aedfc1-33da-4413-a98c-92a4a954a4a3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:39.374294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed22ca-85b7-4921-94d2-b81d467842fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:39.374891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed22ca-85b7-4921-94d2-b81d467842fd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:39.375167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed22ca-85b7-4921-94d2-b81d467842fd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:39.375440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed22ca-85b7-4921-94d2-b81d467842fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:39.376203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed22ca-85b7-4921-94d2-b81d467842fd None admin] RBAC: Adding query filter params (role.id=8b01a7ce4ad14b1aac5e3b46f48a967d, scope.project.id=b42836f77f104b648efac765cddf6b93, user.id=b4986a5d3e6c4adf832c1b1ec8dd768a) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:39.376507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed22ca-85b7-4921-94d2-b81d467842fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:39.377943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed22ca-85b7-4921-94d2-b81d467842fd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:39.383539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 656/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 29 06:16:39 2022] GET /identity/v3/role_assignments?user.id=b4986a5d3e6c4adf832c1b1ec8dd768a&scope.project.id=b42836f77f104b648efac765cddf6b93&role.id=8b01a7ce4ad14b1aac5e3b46f48a967d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:41.418856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389d335-3d23-498f-a535-a74ab863b899 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:41.419489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389d335-3d23-498f-a535-a74ab863b899 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:41.419489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389d335-3d23-498f-a535-a74ab863b899 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:41.420780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 632/1288] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:41.425838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff23967-9a58-4dd0-804e-7f719eb87cfe None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:41.426270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff23967-9a58-4dd0-804e-7f719eb87cfe None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:41.426375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff23967-9a58-4dd0-804e-7f719eb87cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:41.445199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bff23967-9a58-4dd0-804e-7f719eb87cfe None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:41.446601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bff23967-9a58-4dd0-804e-7f719eb87cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:41.481048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:41.519616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 657/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:41.541913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389d335-3d23-498f-a535-a74ab863b899 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:41.642032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389d335-3d23-498f-a535-a74ab863b899 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:41.652391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:41.676623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:41.677272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:41.677513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:41.677784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:41.678394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:41.679783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:41.688675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:41.689986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:41.691103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 633/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:16:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:41.695740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff23967-9a58-4dd0-804e-7f719eb87cfe None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:41.793178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff23967-9a58-4dd0-804e-7f719eb87cfe None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:41.828108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020332b5-72e0-42b9-8c5e-99132160dae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:41.828747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020332b5-72e0-42b9-8c5e-99132160dae3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:41.829030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020332b5-72e0-42b9-8c5e-99132160dae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:41.829302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020332b5-72e0-42b9-8c5e-99132160dae3 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:41.830215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020332b5-72e0-42b9-8c5e-99132160dae3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:41.830515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020332b5-72e0-42b9-8c5e-99132160dae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:41.831872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020332b5-72e0-42b9-8c5e-99132160dae3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:41.836521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 658/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 29 06:16:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:41.843009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:41.942374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b51f-e787-45e9-b288-ceedd35f234e None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:41.975561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8135d-33d6-4ed0-bf1b-9df25026dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:41.976168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8135d-33d6-4ed0-bf1b-9df25026dedb None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:41.976478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8135d-33d6-4ed0-bf1b-9df25026dedb None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:41.976903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8135d-33d6-4ed0-bf1b-9df25026dedb None admin] PATH_INFO: `/v3/services` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:41.977571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8135d-33d6-4ed0-bf1b-9df25026dedb None admin] RBAC: Adding query filter params (type=placement) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:41.977877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8135d-33d6-4ed0-bf1b-9df25026dedb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:41.979235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8135d-33d6-4ed0-bf1b-9df25026dedb None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:41.984359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 634/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 29 06:16:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:43.932914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7d0c-e229-45d9-90a2-d03e2e69f3c1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:43.933461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7d0c-e229-45d9-90a2-d03e2e69f3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:43.933461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7d0c-e229-45d9-90a2-d03e2e69f3c1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:43.934813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 659/1293] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:43.939853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7028f08-95d6-4890-a856-193953d442c4 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:43.940233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7028f08-95d6-4890-a856-193953d442c4 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:43.940402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7028f08-95d6-4890-a856-193953d442c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:43.959095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a7028f08-95d6-4890-a856-193953d442c4 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:43.960279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a7028f08-95d6-4890-a856-193953d442c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:43.989071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:44.025067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 635/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 06:16:44.046566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf7d0c-e229-45d9-90a2-d03e2e69f3c1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:44.077970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:44.146247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf7d0c-e229-45d9-90a2-d03e2e69f3c1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:44.178607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69303886-14f9-45d7-9434-0c31657858aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:44.179242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69303886-14f9-45d7-9434-0c31657858aa None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:44.179524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69303886-14f9-45d7-9434-0c31657858aa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:44.179801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69303886-14f9-45d7-9434-0c31657858aa None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:44.180485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69303886-14f9-45d7-9434-0c31657858aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:44.181868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69303886-14f9-45d7-9434-0c31657858aa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:44.195869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 660/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 29 06:16:44 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:16:46.173460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e891f9-ec64-445a-a9af-cad92334603c None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:46.174016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e891f9-ec64-445a-a9af-cad92334603c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:46.174016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e891f9-ec64-445a-a9af-cad92334603c None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:46.175369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 636/1296] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:46.180332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9715dec-aaf4-409d-bb75-a66f8fa1aa26 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:46.180799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9715dec-aaf4-409d-bb75-a66f8fa1aa26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:46.180908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9715dec-aaf4-409d-bb75-a66f8fa1aa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:46.198517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d9715dec-aaf4-409d-bb75-a66f8fa1aa26 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:46.200209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d9715dec-aaf4-409d-bb75-a66f8fa1aa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:46.270867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 661/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:46.292697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e891f9-ec64-445a-a9af-cad92334603c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:46.393286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e891f9-ec64-445a-a9af-cad92334603c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:46.425955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:46.426524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:46.426808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:46.427085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:46.427704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:46.429090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:46.439841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:46.440969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:46.441802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 637/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:16:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:46.447633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9715dec-aaf4-409d-bb75-a66f8fa1aa26 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:46.469978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:46.564318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9715dec-aaf4-409d-bb75-a66f8fa1aa26 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:46.598410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:46.599026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:46.599309 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:46.599588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:46.600255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:46.600583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:46.601948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:46.607910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 662/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 29 06:16:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:46.613930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:46.644641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:46.709959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48142e1-3d92-4d30-bff6-1cfb7a884c22 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:46.742165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de31d9c-592c-4a37-8a5c-a2cc4e886bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:46.742779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de31d9c-592c-4a37-8a5c-a2cc4e886bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:46.743058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de31d9c-592c-4a37-8a5c-a2cc4e886bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:46.743332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de31d9c-592c-4a37-8a5c-a2cc4e886bc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:46.744066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de31d9c-592c-4a37-8a5c-a2cc4e886bc1 None admin] RBAC: Adding query filter params (interface=public, service_id=35cc40c5827046a78edb48999a7eaa8f, region_id=RegionOne) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:46.744373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de31d9c-592c-4a37-8a5c-a2cc4e886bc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:46.745755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de31d9c-592c-4a37-8a5c-a2cc4e886bc1 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:46.751494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 638/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 29 06:16:46 2022] GET /identity/v3/endpoints?service_id=35cc40c5827046a78edb48999a7eaa8f&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:46.757263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:46.836668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:46.854761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea593d-3f20-497b-9c1f-46366c30eb4c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:46.887219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7459f4-0074-482c-8840-69d6922e29bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:46.887980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7459f4-0074-482c-8840-69d6922e29bd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:46.888264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7459f4-0074-482c-8840-69d6922e29bd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:46.888553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7459f4-0074-482c-8840-69d6922e29bd None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:46.889211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7459f4-0074-482c-8840-69d6922e29bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:46.890571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7459f4-0074-482c-8840-69d6922e29bd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:46.897198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 663/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 29 06:16:46 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:16:48.924886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41755-ff12-49f6-a1a9-a5bb0a1c22a1 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:48.925427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41755-ff12-49f6-a1a9-a5bb0a1c22a1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:48.925427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41755-ff12-49f6-a1a9-a5bb0a1c22a1 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:48.926764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 639/1302] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:16:48.931748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0f8b-c4fb-41e7-864c-60989ac2e9b2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:48.932129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0f8b-c4fb-41e7-864c-60989ac2e9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:48.932299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0f8b-c4fb-41e7-864c-60989ac2e9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:48.950561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3c8c0f8b-c4fb-41e7-864c-60989ac2e9b2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:16:48.951719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3c8c0f8b-c4fb-41e7-864c-60989ac2e9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:16:49.016296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 664/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:16:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 06:16:49.038834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41755-ff12-49f6-a1a9-a5bb0a1c22a1 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:49.130568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:49.137952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41755-ff12-49f6-a1a9-a5bb0a1c22a1 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:49.170083 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:49.170699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:49.170978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:49.171255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:49.171865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:49.173235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:16:49.181715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:16:49.182954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:16:49.184329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 640/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 29 06:16:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 06:16:49.188851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0f8b-c4fb-41e7-864c-60989ac2e9b2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:49.262754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:49.285410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0f8b-c4fb-41e7-864c-60989ac2e9b2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:49.318087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0bc7d-5440-4812-9753-e3036c8c3446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:49.318699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0bc7d-5440-4812-9753-e3036c8c3446 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:49.318984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0bc7d-5440-4812-9753-e3036c8c3446 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:49.319264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0bc7d-5440-4812-9753-e3036c8c3446 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:49.319928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0bc7d-5440-4812-9753-e3036c8c3446 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:16:49.320237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0bc7d-5440-4812-9753-e3036c8c3446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:49.321638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0bc7d-5440-4812-9753-e3036c8c3446 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:49.327308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 665/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 29 06:16:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:16:49.333223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:16:49.428379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c30166-1cf0-4254-9eb8-822a8aca0553 None admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:16:49.463322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa7260-a1cf-4e3f-9ca7-2ed7b6dcdbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:16:49.463933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7260-a1cf-4e3f-9ca7-2ed7b6dcdbd9 None admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:16:49.464212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7260-a1cf-4e3f-9ca7-2ed7b6dcdbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:16:49.464485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7260-a1cf-4e3f-9ca7-2ed7b6dcdbd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:16:49.465232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa7260-a1cf-4e3f-9ca7-2ed7b6dcdbd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:16:49.466575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa7260-a1cf-4e3f-9ca7-2ed7b6dcdbd9 None admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:16:49.485421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 641/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:16:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:16:59.323728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=30 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:16:59.481201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:17:52.223526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3f70-953d-4d16-bc24-89f84e33943f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:52.224673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3f70-953d-4d16-bc24-89f84e33943f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:52.225190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3f70-953d-4d16-bc24-89f84e33943f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:52.231539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 666/1307] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:17:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:17:52.232931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b2a4b-9a21-49a5-aeb2-c10fee4d9f48 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:52.233387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b2a4b-9a21-49a5-aeb2-c10fee4d9f48 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:52.233476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b2a4b-9a21-49a5-aeb2-c10fee4d9f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:52.343062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f41b2a4b-9a21-49a5-aeb2-c10fee4d9f48 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:17:52.344555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f41b2a4b-9a21-49a5-aeb2-c10fee4d9f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:52.522016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 642/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:17:52.558639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fcb19-f71c-4728-b72f-3f8f9a3cc81c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:52.558965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fcb19-f71c-4728-b72f-3f8f9a3cc81c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:52.559306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fcb19-f71c-4728-b72f-3f8f9a3cc81c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:52.560738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 667/1309] 10.222.0.33 () {54 vars in 965 bytes} [Fri Jul 29 06:17:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:17:52.577603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffa03-0435-474a-8e9e-bb3b23c9cabe None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:52.577984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffa03-0435-474a-8e9e-bb3b23c9cabe None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:52.578465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffa03-0435-474a-8e9e-bb3b23c9cabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:52.723849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-fa3ffa03-0435-474a-8e9e-bb3b23c9cabe None None] MFA Rules not processed for user `edf0a00538cf4e10b3bf266f48ea8cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:17:52.725282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fa3ffa03-0435-474a-8e9e-bb3b23c9cabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:52.751461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:17:52.837055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 643/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Fri Jul 29 06:17:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:17:52.839517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fcb19-f71c-4728-b72f-3f8f9a3cc81c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:17:52.841931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d7fcb19-f71c-4728-b72f-3f8f9a3cc81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:52.973743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fcb19-f71c-4728-b72f-3f8f9a3cc81c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:17:53.051828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00381426-66d8-439b-b66e-fe0353bba514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:17:53.052364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00381426-66d8-439b-b66e-fe0353bba514 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:53.052669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00381426-66d8-439b-b66e-fe0353bba514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:53.052947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00381426-66d8-439b-b66e-fe0353bba514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:53.054556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-00381426-66d8-439b-b66e-fe0353bba514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:53.113043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:17:53.175267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00381426-66d8-439b-b66e-fe0353bba514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:17:53.176882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00381426-66d8-439b-b66e-fe0353bba514 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:17:53.198753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 668/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:17:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:17:55.500692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a9771-e4c5-48a2-95a8-0af1c96be5e1 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:55.501275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a9771-e4c5-48a2-95a8-0af1c96be5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:55.501275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a9771-e4c5-48a2-95a8-0af1c96be5e1 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:55.502753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 644/1312] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:17:55.507977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1c0c-d38c-4022-a18c-632f589986ca None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:55.508401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1c0c-d38c-4022-a18c-632f589986ca None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:55.508516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1c0c-d38c-4022-a18c-632f589986ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:55.583366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b38e1c0c-d38c-4022-a18c-632f589986ca None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:17:55.584624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b38e1c0c-d38c-4022-a18c-632f589986ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:55.702420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 669/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:17:55.741169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a9771-e4c5-48a2-95a8-0af1c96be5e1 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:17:55.742736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-958a9771-e4c5-48a2-95a8-0af1c96be5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:55.857716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a9771-e4c5-48a2-95a8-0af1c96be5e1 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:17:55.937144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:17:55.939850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067d612-3883-4ac9-ab24-226963aa38b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:17:55.940831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067d612-3883-4ac9-ab24-226963aa38b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:55.941485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067d612-3883-4ac9-ab24-226963aa38b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:55.941906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067d612-3883-4ac9-ab24-226963aa38b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:55.944109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b067d612-3883-4ac9-ab24-226963aa38b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:56.060061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067d612-3883-4ac9-ab24-226963aa38b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:17:56.061560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067d612-3883-4ac9-ab24-226963aa38b7 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:17:56.081193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 645/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:17:58.390308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d194d-d2c0-411c-a9a9-ea18d542ea10 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:58.390896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d194d-d2c0-411c-a9a9-ea18d542ea10 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:58.390896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d194d-d2c0-411c-a9a9-ea18d542ea10 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:58.392210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 670/1315] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:17:58.397458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244046d-e696-41bb-97dd-f00320a54d69 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:58.397887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244046d-e696-41bb-97dd-f00320a54d69 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:58.398017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244046d-e696-41bb-97dd-f00320a54d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:58.465017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2244046d-e696-41bb-97dd-f00320a54d69 None None] MFA Rules not processed for user `1829d1e77c554b12a2c6482e57c9fe41`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:17:58.466220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2244046d-e696-41bb-97dd-f00320a54d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:58.517513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:17:58.563145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 646/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:17:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:17:58.603919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d194d-d2c0-411c-a9a9-ea18d542ea10 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:17:58.605530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-511d194d-d2c0-411c-a9a9-ea18d542ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:58.615172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:17:58.709311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d194d-d2c0-411c-a9a9-ea18d542ea10 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:17:58.782743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0961f34-3fbd-4806-a6ad-870078f6aba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:17:58.783363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961f34-3fbd-4806-a6ad-870078f6aba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:17:58.783650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961f34-3fbd-4806-a6ad-870078f6aba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:17:58.783933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961f34-3fbd-4806-a6ad-870078f6aba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:17:58.785578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a0961f34-3fbd-4806-a6ad-870078f6aba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:17:58.848170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:17:58.894920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0961f34-3fbd-4806-a6ad-870078f6aba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:17:58.896609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0961f34-3fbd-4806-a6ad-870078f6aba6 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:17:58.917711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 671/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:17:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:01.463828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cd18c-311d-4243-9318-4001ce6f1139 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:01.464396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cd18c-311d-4243-9318-4001ce6f1139 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:01.464396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cd18c-311d-4243-9318-4001ce6f1139 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:01.465803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 647/1318] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:01.470890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c616f-099a-4194-88d3-3ea237562ced None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:01.471317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c616f-099a-4194-88d3-3ea237562ced None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:01.471456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c616f-099a-4194-88d3-3ea237562ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:01.536851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f11c616f-099a-4194-88d3-3ea237562ced None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:01.538068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f11c616f-099a-4194-88d3-3ea237562ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:01.635352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 672/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:01.674218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cd18c-311d-4243-9318-4001ce6f1139 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:01.675767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0a8cd18c-311d-4243-9318-4001ce6f1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:01.780851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cd18c-311d-4243-9318-4001ce6f1139 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:01.852773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ea8b8-f260-4be4-8f95-dc39baa38aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:01.853515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ea8b8-f260-4be4-8f95-dc39baa38aac service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:01.853794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ea8b8-f260-4be4-8f95-dc39baa38aac service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:01.854069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ea8b8-f260-4be4-8f95-dc39baa38aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:01.855667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e93ea8b8-f260-4be4-8f95-dc39baa38aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:01.862952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:01.961490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ea8b8-f260-4be4-8f95-dc39baa38aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:01.962931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ea8b8-f260-4be4-8f95-dc39baa38aac service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:01.981076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 648/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:04.621875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46139603-45a0-4eb3-aefd-7e8754003417 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:04.622420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46139603-45a0-4eb3-aefd-7e8754003417 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:04.622544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46139603-45a0-4eb3-aefd-7e8754003417 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:04.623849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 673/1321] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:04.629034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446f1d5-27e2-4587-8131-2e2a7f73a102 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:04.629448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446f1d5-27e2-4587-8131-2e2a7f73a102 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:04.629584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446f1d5-27e2-4587-8131-2e2a7f73a102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:04.694755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5446f1d5-27e2-4587-8131-2e2a7f73a102 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:04.696515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5446f1d5-27e2-4587-8131-2e2a7f73a102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:04.755447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:04.794521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 649/1322] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:04.830877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46139603-45a0-4eb3-aefd-7e8754003417 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:04.832469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46139603-45a0-4eb3-aefd-7e8754003417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:04.855207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:04.945427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46139603-45a0-4eb3-aefd-7e8754003417 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:05.024927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b5b66-695d-4467-83ed-b7c380b34b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:05.025566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b5b66-695d-4467-83ed-b7c380b34b11 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:05.025854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b5b66-695d-4467-83ed-b7c380b34b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:05.026134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b5b66-695d-4467-83ed-b7c380b34b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:05.027869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a09b5b66-695d-4467-83ed-b7c380b34b11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:05.099782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:05.139526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b5b66-695d-4467-83ed-b7c380b34b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:05.141017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b5b66-695d-4467-83ed-b7c380b34b11 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:05.159130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 674/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:07.235074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c25b5-a1cf-4e0b-8cae-84fddfda199d None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:07.235599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c25b5-a1cf-4e0b-8cae-84fddfda199d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:07.235680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c25b5-a1cf-4e0b-8cae-84fddfda199d None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:07.237031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 650/1324] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:07.242521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda633d-adf6-47fe-abfe-68094adb3720 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:07.242521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda633d-adf6-47fe-abfe-68094adb3720 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:07.243462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda633d-adf6-47fe-abfe-68094adb3720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:07.311026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7eda633d-adf6-47fe-abfe-68094adb3720 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:07.312221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7eda633d-adf6-47fe-abfe-68094adb3720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:07.408191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 675/1325] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:07.444806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c25b5-a1cf-4e0b-8cae-84fddfda199d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:07.446643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-094c25b5-a1cf-4e0b-8cae-84fddfda199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:07.552165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c25b5-a1cf-4e0b-8cae-84fddfda199d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:07.628891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a259d4-a08c-465b-b8de-2176ba08fb1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:07.629519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a259d4-a08c-465b-b8de-2176ba08fb1e service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:07.629804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a259d4-a08c-465b-b8de-2176ba08fb1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:07.630082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a259d4-a08c-465b-b8de-2176ba08fb1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:07.631714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-48a259d4-a08c-465b-b8de-2176ba08fb1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:07.648786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:07.747995 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a259d4-a08c-465b-b8de-2176ba08fb1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:07.749494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a259d4-a08c-465b-b8de-2176ba08fb1e service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:07.770576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 651/1326] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:09.822951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff79355-9273-448e-84a6-458169968693 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:09.823518 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff79355-9273-448e-84a6-458169968693 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:09.823518 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff79355-9273-448e-84a6-458169968693 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:09.824910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 676/1327] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:09.830192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417092ce-5266-4936-90ad-be51f049c837 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:09.830634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417092ce-5266-4936-90ad-be51f049c837 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:09.830869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417092ce-5266-4936-90ad-be51f049c837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:09.902561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-417092ce-5266-4936-90ad-be51f049c837 None None] MFA Rules not processed for user `1829d1e77c554b12a2c6482e57c9fe41`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:09.903777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-417092ce-5266-4936-90ad-be51f049c837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:09.969970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:10.002350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 652/1328] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:09 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:10.040085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff79355-9273-448e-84a6-458169968693 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:10.041741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bff79355-9273-448e-84a6-458169968693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:10.076023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:10.155988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff79355-9273-448e-84a6-458169968693 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:10.235324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b68a60-cbb9-41e4-bcb4-74759d87fa34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:10.235946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b68a60-cbb9-41e4-bcb4-74759d87fa34 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:10.236228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b68a60-cbb9-41e4-bcb4-74759d87fa34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:10.236503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b68a60-cbb9-41e4-bcb4-74759d87fa34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:10.238141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d1b68a60-cbb9-41e4-bcb4-74759d87fa34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:10.317359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:10.349673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b68a60-cbb9-41e4-bcb4-74759d87fa34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:10.351116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b68a60-cbb9-41e4-bcb4-74759d87fa34 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:10.370279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 677/1329] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:10 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:12.631591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4883e-d557-4cc3-8fbe-cf3ed4f240fe None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:12.632151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4883e-d557-4cc3-8fbe-cf3ed4f240fe None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:12.632151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4883e-d557-4cc3-8fbe-cf3ed4f240fe None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:12.634081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 653/1330] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:12.639114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1855e6-baf9-4fbd-99c6-0691da382ea1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:12.639509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1855e6-baf9-4fbd-99c6-0691da382ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:12.639682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1855e6-baf9-4fbd-99c6-0691da382ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:12.707136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8b1855e6-baf9-4fbd-99c6-0691da382ea1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:12.708358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8b1855e6-baf9-4fbd-99c6-0691da382ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:12.808684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 678/1331] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:12.841723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4883e-d557-4cc3-8fbe-cf3ed4f240fe None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:12.843243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-35c4883e-d557-4cc3-8fbe-cf3ed4f240fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:12.944515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4883e-d557-4cc3-8fbe-cf3ed4f240fe None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:13.016939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0f003-48a7-47cf-a6df-61a532c7468a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:13.017554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f003-48a7-47cf-a6df-61a532c7468a service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:13.017833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f003-48a7-47cf-a6df-61a532c7468a service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:13.018105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f003-48a7-47cf-a6df-61a532c7468a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:13.019679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d8d0f003-48a7-47cf-a6df-61a532c7468a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:13.046861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:13.131658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0f003-48a7-47cf-a6df-61a532c7468a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:13.133260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0f003-48a7-47cf-a6df-61a532c7468a service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:13.151702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 654/1332] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:15.216609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e7273-6022-42a4-9186-cc3a42150084 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:15.217183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e7273-6022-42a4-9186-cc3a42150084 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:15.217183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e7273-6022-42a4-9186-cc3a42150084 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:15.218509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 679/1333] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:18:15.223657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431e9e2-4782-49de-9eea-d3b2736b2baa None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:15.224094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431e9e2-4782-49de-9eea-d3b2736b2baa None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:15.224214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431e9e2-4782-49de-9eea-d3b2736b2baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:15.290947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-4431e9e2-4782-49de-9eea-d3b2736b2baa None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:15.292161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4431e9e2-4782-49de-9eea-d3b2736b2baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:15.368790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:15.391462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 655/1334] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:15.431764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e7273-6022-42a4-9186-cc3a42150084 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:15.433365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-683e7273-6022-42a4-9186-cc3a42150084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:15.473121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:15.535791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e7273-6022-42a4-9186-cc3a42150084 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:15.609711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ec148-d37c-4f48-b027-5e39ebc7594e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:15.610322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ec148-d37c-4f48-b027-5e39ebc7594e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:15.610601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ec148-d37c-4f48-b027-5e39ebc7594e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:15.610871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ec148-d37c-4f48-b027-5e39ebc7594e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:15.612457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0d3ec148-d37c-4f48-b027-5e39ebc7594e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:15.695230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:15.721721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ec148-d37c-4f48-b027-5e39ebc7594e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:15.723209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ec148-d37c-4f48-b027-5e39ebc7594e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:15.741940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 680/1335] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:18:18.056310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725c14a-ca55-4068-8e32-b2c18d64b4c8 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:18.056781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725c14a-ca55-4068-8e32-b2c18d64b4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:18.056973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725c14a-ca55-4068-8e32-b2c18d64b4c8 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:18.058337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 656/1336] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:18.063339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88564028-a2d3-4142-af09-70c8d356dc80 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:18.063785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88564028-a2d3-4142-af09-70c8d356dc80 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:18.063889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88564028-a2d3-4142-af09-70c8d356dc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:18.128602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-88564028-a2d3-4142-af09-70c8d356dc80 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:18.129790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-88564028-a2d3-4142-af09-70c8d356dc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:18.225541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 681/1337] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:18.261384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725c14a-ca55-4068-8e32-b2c18d64b4c8 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:18.262910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a725c14a-ca55-4068-8e32-b2c18d64b4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:18.365222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725c14a-ca55-4068-8e32-b2c18d64b4c8 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:18.438563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4a09-9257-4d67-a527-993129170366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:18.439189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4a09-9257-4d67-a527-993129170366 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:18.439478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4a09-9257-4d67-a527-993129170366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:18.439757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4a09-9257-4d67-a527-993129170366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:18.441397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b5cc4a09-9257-4d67-a527-993129170366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:18.481192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:18.553204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc4a09-9257-4d67-a527-993129170366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:18.554676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc4a09-9257-4d67-a527-993129170366 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:18.573168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 657/1338] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:22.492816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db59060-6e53-4028-8dce-6889d4c87ac0 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:22.493421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db59060-6e53-4028-8dce-6889d4c87ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:22.493421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db59060-6e53-4028-8dce-6889d4c87ac0 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:22.494828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 682/1339] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:22.500091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bdade-ece1-4a8f-a1ca-3d209c12d480 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:22.500499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bdade-ece1-4a8f-a1ca-3d209c12d480 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:22.500816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bdade-ece1-4a8f-a1ca-3d209c12d480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:22.569124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-311bdade-ece1-4a8f-a1ca-3d209c12d480 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:22.570787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-311bdade-ece1-4a8f-a1ca-3d209c12d480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:22.659524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:22.671556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 658/1340] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:22.711277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db59060-6e53-4028-8dce-6889d4c87ac0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:22.713043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4db59060-6e53-4028-8dce-6889d4c87ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:22.766157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:22.828030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db59060-6e53-4028-8dce-6889d4c87ac0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:22.903695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f6c2b-ee67-4ab7-8100-59be2edfab2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:22.904328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f6c2b-ee67-4ab7-8100-59be2edfab2d service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:22.904639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f6c2b-ee67-4ab7-8100-59be2edfab2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:22.904921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f6c2b-ee67-4ab7-8100-59be2edfab2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:22.906546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-081f6c2b-ee67-4ab7-8100-59be2edfab2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:23.000437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:23.017430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f6c2b-ee67-4ab7-8100-59be2edfab2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:23.019008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f6c2b-ee67-4ab7-8100-59be2edfab2d service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:23.037488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 683/1341] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:25.696612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4ced-7446-497b-b9c6-15c885f8a3b2 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:25.697216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4ced-7446-497b-b9c6-15c885f8a3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:25.697216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4ced-7446-497b-b9c6-15c885f8a3b2 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:25.698649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 659/1342] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:25.704922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011b6d9-2437-4548-8540-e0c9f476fe70 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:25.705476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011b6d9-2437-4548-8540-e0c9f476fe70 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:25.705581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011b6d9-2437-4548-8540-e0c9f476fe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:25.780595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5011b6d9-2437-4548-8540-e0c9f476fe70 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:25.781848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5011b6d9-2437-4548-8540-e0c9f476fe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:25.884178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 684/1343] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:25.933515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ed02-16e5-4f71-a255-4b604d27e8b1 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:25.933897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ed02-16e5-4f71-a255-4b604d27e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:25.934244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12ed02-16e5-4f71-a255-4b604d27e8b1 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:25.935730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 660/1344] 10.222.0.33 () {54 vars in 965 bytes} [Fri Jul 29 06:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:25.941486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbb73b-8c0e-4acf-ad7b-4fc977162031 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:25.941787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbb73b-8c0e-4acf-ad7b-4fc977162031 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:25.942071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbb73b-8c0e-4acf-ad7b-4fc977162031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:26.010332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:26.020014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b1bbb73b-8c0e-4acf-ad7b-4fc977162031 None None] MFA Rules not processed for user `edf0a00538cf4e10b3bf266f48ea8cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:26.021262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b1bbb73b-8c0e-4acf-ad7b-4fc977162031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:26.099020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 685/1345] 10.222.0.33 () {58 vars in 1062 bytes} [Fri Jul 29 06:18:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:26.108795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12ed02-16e5-4f71-a255-4b604d27e8b1 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:26.110328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9b12ed02-16e5-4f71-a255-4b604d27e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:26.217314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12ed02-16e5-4f71-a255-4b604d27e8b1 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:26.293596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a2b9-1291-4d19-8b2a-12be32564439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:26.294143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a2b9-1291-4d19-8b2a-12be32564439 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:26.294429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a2b9-1291-4d19-8b2a-12be32564439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:26.294707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a2b9-1291-4d19-8b2a-12be32564439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:26.296319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9fa4a2b9-1291-4d19-8b2a-12be32564439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:26.341702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:26.410155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4a2b9-1291-4d19-8b2a-12be32564439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:26.411595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4a2b9-1291-4d19-8b2a-12be32564439 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:26.433387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 661/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:31.198009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb4106-87da-469f-9070-356c51c944b8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:31.198515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb4106-87da-469f-9070-356c51c944b8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:31.198627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb4106-87da-469f-9070-356c51c944b8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:31.200066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 686/1347] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:31.205612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c97a-338d-4093-9376-b5e9943eb9d6 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:31.206058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c97a-338d-4093-9376-b5e9943eb9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:31.206153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c97a-338d-4093-9376-b5e9943eb9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:31.273287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8547c97a-338d-4093-9376-b5e9943eb9d6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:31.274587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8547c97a-338d-4093-9376-b5e9943eb9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:31.372904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:31.375585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 662/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:31.413139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb4106-87da-469f-9070-356c51c944b8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:31.415238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40cb4106-87da-469f-9070-356c51c944b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:31.519804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb4106-87da-469f-9070-356c51c944b8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:31.567193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:31.593858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18210b69-c996-4c8b-9257-9add5d12f822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:31.594480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18210b69-c996-4c8b-9257-9add5d12f822 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:31.594770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18210b69-c996-4c8b-9257-9add5d12f822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:31.595050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18210b69-c996-4c8b-9257-9add5d12f822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:31.596702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-18210b69-c996-4c8b-9257-9add5d12f822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:31.703532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18210b69-c996-4c8b-9257-9add5d12f822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:31.705210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18210b69-c996-4c8b-9257-9add5d12f822 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:31.723959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 687/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:36.615326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f335db3-f219-4b31-b4aa-ffb2b6633a90 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:36.615832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f335db3-f219-4b31-b4aa-ffb2b6633a90 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:36.615938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f335db3-f219-4b31-b4aa-ffb2b6633a90 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:36.618464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 663/1350] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:36.623705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62dd38-03ec-419d-ba68-280d6b97884d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:36.624123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62dd38-03ec-419d-ba68-280d6b97884d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:36.624267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62dd38-03ec-419d-ba68-280d6b97884d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:36.757622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9c62dd38-03ec-419d-ba68-280d6b97884d None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:36.760177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9c62dd38-03ec-419d-ba68-280d6b97884d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:36.783936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:36.909186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 688/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:18:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:40.972378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3329ae-46c7-45b8-b687-e97440fa5d7c None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:40.972946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3329ae-46c7-45b8-b687-e97440fa5d7c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:40.973433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3329ae-46c7-45b8-b687-e97440fa5d7c None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:40.975978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 664/1352] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:18:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:40.982301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441738-2352-479c-b0f8-38e6f7a934e1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:40.983425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441738-2352-479c-b0f8-38e6f7a934e1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:40.983425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441738-2352-479c-b0f8-38e6f7a934e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:41.085949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5f441738-2352-479c-b0f8-38e6f7a934e1 None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:41.087182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5f441738-2352-479c-b0f8-38e6f7a934e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:41.160427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:41.199398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 689/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:18:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:42.757196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5100f2-8135-4cce-afdb-7cdc22d0d10f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:42.757705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5100f2-8135-4cce-afdb-7cdc22d0d10f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:42.757817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5100f2-8135-4cce-afdb-7cdc22d0d10f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:42.759249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 665/1354] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:42.764482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329feb5-a51f-4bc5-8471-a15682990036 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:42.765002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329feb5-a51f-4bc5-8471-a15682990036 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:42.765115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329feb5-a51f-4bc5-8471-a15682990036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:42.842133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7329feb5-a51f-4bc5-8471-a15682990036 None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:42.843355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7329feb5-a51f-4bc5-8471-a15682990036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:42.959470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 690/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:18:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:18:46.207335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fddbb-1bf3-48fa-b4e7-29b47781ba8f None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:46.207751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fddbb-1bf3-48fa-b4e7-29b47781ba8f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:46.207751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fddbb-1bf3-48fa-b4e7-29b47781ba8f None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:46.212148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 666/1356] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:46.217944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfedaf8-c706-4233-8bc8-cfedf8576d20 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:46.218439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfedaf8-c706-4233-8bc8-cfedf8576d20 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:46.218550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfedaf8-c706-4233-8bc8-cfedf8576d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:46.261603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:46.300605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bdfedaf8-c706-4233-8bc8-cfedf8576d20 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:46.301876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bdfedaf8-c706-4233-8bc8-cfedf8576d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:46.416585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 691/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:46.474300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93f6c-f743-44fe-837a-14df471ee9fe None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:46.474702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93f6c-f743-44fe-837a-14df471ee9fe None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:46.475088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93f6c-f743-44fe-837a-14df471ee9fe None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:46.477020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 667/1358] 10.222.0.33 () {54 vars in 962 bytes} [Fri Jul 29 06:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:46.486927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef3f9-c4de-4930-a87e-50948f595a26 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:46.487647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef3f9-c4de-4930-a87e-50948f595a26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:46.488090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef3f9-c4de-4930-a87e-50948f595a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:46.572082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-db3ef3f9-c4de-4930-a87e-50948f595a26 None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:46.573198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-db3ef3f9-c4de-4930-a87e-50948f595a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:46.627479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:46.703955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 692/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Fri Jul 29 06:18:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:46.712671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93f6c-f743-44fe-837a-14df471ee9fe None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:46.714573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-22e93f6c-f743-44fe-837a-14df471ee9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:46.875837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93f6c-f743-44fe-837a-14df471ee9fe None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:46.955790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:46.998920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844decc-9c78-4c1d-a22a-5d03a75339e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:47.000073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844decc-9c78-4c1d-a22a-5d03a75339e2 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:47.000710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844decc-9c78-4c1d-a22a-5d03a75339e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:47.001307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844decc-9c78-4c1d-a22a-5d03a75339e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:47.004629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9844decc-9c78-4c1d-a22a-5d03a75339e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:47.119495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844decc-9c78-4c1d-a22a-5d03a75339e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:47.121506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844decc-9c78-4c1d-a22a-5d03a75339e2 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:47.144661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 668/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:49.400703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14712a5-4112-41c2-8d44-c14ba2f34af1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:49.401314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14712a5-4112-41c2-8d44-c14ba2f34af1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:49.401314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14712a5-4112-41c2-8d44-c14ba2f34af1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:49.402757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 693/1361] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:49.407584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75270ae-cdb4-4c1b-8b36-60102d513828 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:49.407994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75270ae-cdb4-4c1b-8b36-60102d513828 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:49.408172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75270ae-cdb4-4c1b-8b36-60102d513828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:49.611303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d75270ae-cdb4-4c1b-8b36-60102d513828 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:49.612499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d75270ae-cdb4-4c1b-8b36-60102d513828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:49.653464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:49.728589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 669/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:49.778036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14712a5-4112-41c2-8d44-c14ba2f34af1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:49.779979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a14712a5-4112-41c2-8d44-c14ba2f34af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:49.977917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14712a5-4112-41c2-8d44-c14ba2f34af1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:49.988401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:50.107274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f66ba5-cb82-48e6-b653-59a7feb0ef3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:50.107904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f66ba5-cb82-48e6-b653-59a7feb0ef3f service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:50.108190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f66ba5-cb82-48e6-b653-59a7feb0ef3f service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:50.108464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f66ba5-cb82-48e6-b653-59a7feb0ef3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:50.110106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d8f66ba5-cb82-48e6-b653-59a7feb0ef3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:50.210410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f66ba5-cb82-48e6-b653-59a7feb0ef3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:50.211934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f66ba5-cb82-48e6-b653-59a7feb0ef3f service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:50.232068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 694/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:52.613847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2dc7-9c3d-440d-a49f-119eeae25b60 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:52.614704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2dc7-9c3d-440d-a49f-119eeae25b60 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:52.615129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2dc7-9c3d-440d-a49f-119eeae25b60 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:52.617528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 670/1364] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:52.623221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b106447-038f-42b0-857b-03a9b5370a1a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:52.623691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b106447-038f-42b0-857b-03a9b5370a1a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:52.624050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b106447-038f-42b0-857b-03a9b5370a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:52.640916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:52.698541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7b106447-038f-42b0-857b-03a9b5370a1a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:52.699922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7b106447-038f-42b0-857b-03a9b5370a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:52.816815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 695/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:52.863583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9422c28-9cca-408a-a34d-fd8d597b9b24 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:52.863925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9422c28-9cca-408a-a34d-fd8d597b9b24 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:52.864145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9422c28-9cca-408a-a34d-fd8d597b9b24 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:52.866416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 671/1366] 10.222.0.33 () {54 vars in 962 bytes} [Fri Jul 29 06:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:52.872249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06547c3-28d7-4d48-8453-365e26e3bd5b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:52.872775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06547c3-28d7-4d48-8453-365e26e3bd5b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:52.873064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06547c3-28d7-4d48-8453-365e26e3bd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:52.956015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f06547c3-28d7-4d48-8453-365e26e3bd5b None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:52.957410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f06547c3-28d7-4d48-8453-365e26e3bd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:52.999262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:53.093246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 696/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Fri Jul 29 06:18:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:53.103858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9422c28-9cca-408a-a34d-fd8d597b9b24 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:53.105600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d9422c28-9cca-408a-a34d-fd8d597b9b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:53.297652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9422c28-9cca-408a-a34d-fd8d597b9b24 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:53.323019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:53.440567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859b437-019d-4990-928d-fdc4ccbfd12d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:53.441336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b437-019d-4990-928d-fdc4ccbfd12d service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:53.441747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b437-019d-4990-928d-fdc4ccbfd12d service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:53.442122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b437-019d-4990-928d-fdc4ccbfd12d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:53.445404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8859b437-019d-4990-928d-fdc4ccbfd12d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:53.563416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859b437-019d-4990-928d-fdc4ccbfd12d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:53.565298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859b437-019d-4990-928d-fdc4ccbfd12d service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:53.588224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 672/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:54.538607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79e50-34a4-4947-ad52-443d8e7e10ca None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:54.539185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79e50-34a4-4947-ad52-443d8e7e10ca None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:54.539185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79e50-34a4-4947-ad52-443d8e7e10ca None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:54.540325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 697/1369] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:54.545782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356687d-bf7b-4846-83ef-73743fb7fb73 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:54.546197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356687d-bf7b-4846-83ef-73743fb7fb73 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:54.546565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356687d-bf7b-4846-83ef-73743fb7fb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:54.580317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:54.619597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2356687d-bf7b-4846-83ef-73743fb7fb73 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:54.621668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2356687d-bf7b-4846-83ef-73743fb7fb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:54.738899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 673/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:55.050900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004be8e-b355-4873-912c-12beb890d72e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:55.051286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004be8e-b355-4873-912c-12beb890d72e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:55.051645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004be8e-b355-4873-912c-12beb890d72e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:55.053435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 698/1371] 10.222.0.33 () {54 vars in 964 bytes} [Fri Jul 29 06:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:55.058550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251928a-d8e9-4eb3-880e-1e80541ac82c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:55.058851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251928a-d8e9-4eb3-880e-1e80541ac82c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:55.059133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251928a-d8e9-4eb3-880e-1e80541ac82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:55.146139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5251928a-d8e9-4eb3-880e-1e80541ac82c None None] MFA Rules not processed for user `ec1feaf015a847ffa719fd3a53de02e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:55.148968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5251928a-d8e9-4eb3-880e-1e80541ac82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:55.205234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:55.239344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 674/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Fri Jul 29 06:18:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:55.249063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004be8e-b355-4873-912c-12beb890d72e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:55.250650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4004be8e-b355-4873-912c-12beb890d72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:55.366726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004be8e-b355-4873-912c-12beb890d72e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:55.410956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:55.452404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96578ff3-39d7-4918-bd89-9221c3fa9f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:55.453046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96578ff3-39d7-4918-bd89-9221c3fa9f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:55.453356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96578ff3-39d7-4918-bd89-9221c3fa9f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:55.453645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96578ff3-39d7-4918-bd89-9221c3fa9f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:55.455798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-96578ff3-39d7-4918-bd89-9221c3fa9f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:55.579633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96578ff3-39d7-4918-bd89-9221c3fa9f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:55.581066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96578ff3-39d7-4918-bd89-9221c3fa9f68 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:55.608320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 699/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:18:55.737323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7cd64-4c73-4cfa-a9a6-68d357712236 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:55.737737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7cd64-4c73-4cfa-a9a6-68d357712236 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:55.737887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7cd64-4c73-4cfa-a9a6-68d357712236 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:55.739261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 675/1374] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:55.745346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f212f3-e9aa-483a-9548-1f282b0f0733 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:55.745663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f212f3-e9aa-483a-9548-1f282b0f0733 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:55.745994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f212f3-e9aa-483a-9548-1f282b0f0733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:55.823105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b0f212f3-e9aa-483a-9548-1f282b0f0733 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:55.824462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b0f212f3-e9aa-483a-9548-1f282b0f0733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:55.848765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:55.965905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 700/1375] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:56.012029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7cd64-4c73-4cfa-a9a6-68d357712236 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:56.013657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-51d7cd64-4c73-4cfa-a9a6-68d357712236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:56.161184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7cd64-4c73-4cfa-a9a6-68d357712236 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:56.214278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:56.279108 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832c7cb-2180-42e3-a62d-1435bc61a4e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:56.279736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832c7cb-2180-42e3-a62d-1435bc61a4e4 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:56.280018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832c7cb-2180-42e3-a62d-1435bc61a4e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:56.280288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832c7cb-2180-42e3-a62d-1435bc61a4e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:56.281956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9832c7cb-2180-42e3-a62d-1435bc61a4e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:56.381717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832c7cb-2180-42e3-a62d-1435bc61a4e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:56.383307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832c7cb-2180-42e3-a62d-1435bc61a4e4 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:56.401804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 676/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:18:58.445564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975616-bfed-4345-9a5b-36840009e6cc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:58.446121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975616-bfed-4345-9a5b-36840009e6cc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:58.446121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975616-bfed-4345-9a5b-36840009e6cc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:58.447469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 701/1377] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:18:58.453615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056f1ab-d6eb-4a75-953d-0f2758d32f6d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:58.454366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056f1ab-d6eb-4a75-953d-0f2758d32f6d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:58.454501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056f1ab-d6eb-4a75-953d-0f2758d32f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:58.525004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3056f1ab-d6eb-4a75-953d-0f2758d32f6d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:18:58.526220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3056f1ab-d6eb-4a75-953d-0f2758d32f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:58.532895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:58.636320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 677/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:18:58.688299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975616-bfed-4345-9a5b-36840009e6cc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:18:58.689952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c9975616-bfed-4345-9a5b-36840009e6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:58.842582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975616-bfed-4345-9a5b-36840009e6cc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:18:58.846302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:18:58.969179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f50498-d35a-4994-8235-6cd1f388a7aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:18:58.969822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f50498-d35a-4994-8235-6cd1f388a7aa service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:18:58.970104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f50498-d35a-4994-8235-6cd1f388a7aa service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:18:58.970364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f50498-d35a-4994-8235-6cd1f388a7aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:18:58.971989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e5f50498-d35a-4994-8235-6cd1f388a7aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:18:59.088853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f50498-d35a-4994-8235-6cd1f388a7aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:18:59.092064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f50498-d35a-4994-8235-6cd1f388a7aa service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:18:59.121974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 702/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:01.193282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71911d7-c9a9-47dc-915c-75d7794759e9 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:01.193812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71911d7-c9a9-47dc-915c-75d7794759e9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:01.194052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71911d7-c9a9-47dc-915c-75d7794759e9 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:01.195503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 678/1380] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:01.201066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e49847-e1dd-4b96-816e-542ac2796360 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:01.201495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e49847-e1dd-4b96-816e-542ac2796360 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:01.201700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e49847-e1dd-4b96-816e-542ac2796360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:01.213955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:01.275105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-70e49847-e1dd-4b96-816e-542ac2796360 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:01.276338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-70e49847-e1dd-4b96-816e-542ac2796360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:01.377785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 703/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:01.422140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71911d7-c9a9-47dc-915c-75d7794759e9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:01.423656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b71911d7-c9a9-47dc-915c-75d7794759e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:01.581185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:01.593073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71911d7-c9a9-47dc-915c-75d7794759e9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:01.715355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555f500-a18e-4960-a57f-63bf5ee57d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:01.715896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555f500-a18e-4960-a57f-63bf5ee57d9b service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:01.716173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555f500-a18e-4960-a57f-63bf5ee57d9b service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:01.716450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555f500-a18e-4960-a57f-63bf5ee57d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:01.720381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e555f500-a18e-4960-a57f-63bf5ee57d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:01.821154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555f500-a18e-4960-a57f-63bf5ee57d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:01.822798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555f500-a18e-4960-a57f-63bf5ee57d9b service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:01.834241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:01.856137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 679/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:03.911190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6293f-40a7-4981-ae1a-c8072a9a3790 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:03.911753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6293f-40a7-4981-ae1a-c8072a9a3790 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:03.911753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6293f-40a7-4981-ae1a-c8072a9a3790 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:03.913346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 704/1383] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:03.918773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91863-4c49-4907-88e1-8d65a30d59fc None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:03.919199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91863-4c49-4907-88e1-8d65a30d59fc None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:03.919340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91863-4c49-4907-88e1-8d65a30d59fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:03.993405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-53b91863-4c49-4907-88e1-8d65a30d59fc None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:03.994591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-53b91863-4c49-4907-88e1-8d65a30d59fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:04.099092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 680/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:04.141708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6293f-40a7-4981-ae1a-c8072a9a3790 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:04.143229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-41e6293f-40a7-4981-ae1a-c8072a9a3790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:04.231531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:04.312954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6293f-40a7-4981-ae1a-c8072a9a3790 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:04.441560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499cca-cdab-4e91-83de-1fe710920c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:04.442194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499cca-cdab-4e91-83de-1fe710920c37 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:04.442478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499cca-cdab-4e91-83de-1fe710920c37 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:04.442758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499cca-cdab-4e91-83de-1fe710920c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:04.444407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dc499cca-cdab-4e91-83de-1fe710920c37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:04.500922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:04.544117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc499cca-cdab-4e91-83de-1fe710920c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:04.546312 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc499cca-cdab-4e91-83de-1fe710920c37 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:04.569402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 705/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:07.045942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679764a6-c46e-49aa-ab0a-3b4996b20418 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:07.046449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679764a6-c46e-49aa-ab0a-3b4996b20418 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:07.046449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679764a6-c46e-49aa-ab0a-3b4996b20418 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:07.047847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 681/1386] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:07.055175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6364ba-1798-4482-b957-4fd3581ea791 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:07.055603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6364ba-1798-4482-b957-4fd3581ea791 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:07.055983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6364ba-1798-4482-b957-4fd3581ea791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:07.127897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-eb6364ba-1798-4482-b957-4fd3581ea791 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:07.129153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eb6364ba-1798-4482-b957-4fd3581ea791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:07.236644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 706/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:19:07.302492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679764a6-c46e-49aa-ab0a-3b4996b20418 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:07.304384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-679764a6-c46e-49aa-ab0a-3b4996b20418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:07.387845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:07.476103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679764a6-c46e-49aa-ab0a-3b4996b20418 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:07.606485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fcc13-68c7-4d60-b5ae-921664e49f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:07.607163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fcc13-68c7-4d60-b5ae-921664e49f36 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:07.607443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fcc13-68c7-4d60-b5ae-921664e49f36 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:07.607715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fcc13-68c7-4d60-b5ae-921664e49f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:07.609384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-466fcc13-68c7-4d60-b5ae-921664e49f36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:07.649969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:07.704586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fcc13-68c7-4d60-b5ae-921664e49f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:07.706171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fcc13-68c7-4d60-b5ae-921664e49f36 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:07.728379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 682/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:09.802242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1588a-1df8-4370-b4ce-f772d8a7cf85 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:09.803333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1588a-1df8-4370-b4ce-f772d8a7cf85 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:09.803485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1588a-1df8-4370-b4ce-f772d8a7cf85 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:09.804818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 707/1389] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:09.810309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ba9bc-4c93-4e16-9055-f1068ec5e6df None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:09.810744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ba9bc-4c93-4e16-9055-f1068ec5e6df None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:09.810857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ba9bc-4c93-4e16-9055-f1068ec5e6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:09.876700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e95ba9bc-4c93-4e16-9055-f1068ec5e6df None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:09.877905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e95ba9bc-4c93-4e16-9055-f1068ec5e6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:09.975543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 683/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:10.018516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1588a-1df8-4370-b4ce-f772d8a7cf85 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:10.020033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-53c1588a-1df8-4370-b4ce-f772d8a7cf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:10.051600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:10.169003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1588a-1df8-4370-b4ce-f772d8a7cf85 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:10.288825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:10.302381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5a15a-4732-46f4-b964-12b9990da83f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:10.303040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5a15a-4732-46f4-b964-12b9990da83f service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:10.303316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5a15a-4732-46f4-b964-12b9990da83f service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:10.303593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5a15a-4732-46f4-b964-12b9990da83f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:10.305289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-05d5a15a-4732-46f4-b964-12b9990da83f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:10.405116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5a15a-4732-46f4-b964-12b9990da83f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:10.406704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5a15a-4732-46f4-b964-12b9990da83f service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:10.426012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 708/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:10.685822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d39e-ebcb-4f44-9745-5df77d2a15a5 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:10.686113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d39e-ebcb-4f44-9745-5df77d2a15a5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:10.686417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d39e-ebcb-4f44-9745-5df77d2a15a5 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:10.687964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 684/1392] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:10.693613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddc3db-f221-45cb-a759-ef5ea3e6f91e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:10.693884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddc3db-f221-45cb-a759-ef5ea3e6f91e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:10.694229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddc3db-f221-45cb-a759-ef5ea3e6f91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:10.766893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c7ddc3db-f221-45cb-a759-ef5ea3e6f91e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:10.768112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c7ddc3db-f221-45cb-a759-ef5ea3e6f91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:10.829941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:10.869091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 709/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:10.923973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f473b-f6fd-4dd8-9543-2f22bb0367e2 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:10.924470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f473b-f6fd-4dd8-9543-2f22bb0367e2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:10.924613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f473b-f6fd-4dd8-9543-2f22bb0367e2 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:10.926052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 685/1394] 10.222.0.33 () {54 vars in 966 bytes} [Fri Jul 29 06:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:10.931960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28933dc6-b614-42b4-9c6e-e6a37e70f726 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:10.932325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28933dc6-b614-42b4-9c6e-e6a37e70f726 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:10.932712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28933dc6-b614-42b4-9c6e-e6a37e70f726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:11.010366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-28933dc6-b614-42b4-9c6e-e6a37e70f726 None None] MFA Rules not processed for user `bc4b8f7545934151aafa986da871b944`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:11.011517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-28933dc6-b614-42b4-9c6e-e6a37e70f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:11.088057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 710/1395] 10.222.0.33 () {58 vars in 1063 bytes} [Fri Jul 29 06:19:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:11.095924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f473b-f6fd-4dd8-9543-2f22bb0367e2 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:11.097463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9e9f473b-f6fd-4dd8-9543-2f22bb0367e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:11.112015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:11.210920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f473b-f6fd-4dd8-9543-2f22bb0367e2 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:11.284462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bae554-3415-4bd6-b743-70b5f35b8dea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4b8f7545934151aafa986da871b944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:11.285028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bae554-3415-4bd6-b743-70b5f35b8dea service glance] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:11.285318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bae554-3415-4bd6-b743-70b5f35b8dea service glance] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:11.285606 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bae554-3415-4bd6-b743-70b5f35b8dea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:11.287789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c7bae554-3415-4bd6-b743-70b5f35b8dea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:11.357361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:11.398513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bae554-3415-4bd6-b743-70b5f35b8dea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:11.399962 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bae554-3415-4bd6-b743-70b5f35b8dea service glance] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:11.419653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 686/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:12.157978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca51822-59b3-420f-be67-837e82943277 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:12.158471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca51822-59b3-420f-be67-837e82943277 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:12.158559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca51822-59b3-420f-be67-837e82943277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:12.228027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0ca51822-59b3-420f-be67-837e82943277 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:12.229281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0ca51822-59b3-420f-be67-837e82943277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:12.238825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:12.317926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 711/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:19:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:12.361054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e4a7-e0a0-41b2-92ce-2fc8a4fdee88 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:12.361532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e4a7-e0a0-41b2-92ce-2fc8a4fdee88 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:12.361628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e4a7-e0a0-41b2-92ce-2fc8a4fdee88 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:12.362967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 687/1398] 10.222.0.33 () {54 vars in 943 bytes} [Fri Jul 29 06:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:12.369481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfde59-63ca-4340-ab8c-939126598894 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:12.369843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfde59-63ca-4340-ab8c-939126598894 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:12.370195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfde59-63ca-4340-ab8c-939126598894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:12.452126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b0cfde59-63ca-4340-ab8c-939126598894 None None] MFA Rules not processed for user `e35b790a4a5b4306a5537984d0b449e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:12.453572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b0cfde59-63ca-4340-ab8c-939126598894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:12.485020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540958d-b8a9-487a-80a2-6e8aabefb2fc None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:12.485356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540958d-b8a9-487a-80a2-6e8aabefb2fc None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:12.485578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540958d-b8a9-487a-80a2-6e8aabefb2fc None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:12.487070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 688/1399] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:12.492466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:12.492859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:12.493145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:12.553834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:12.561926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:12.563137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:12.577058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 712/1400] 10.222.0.33 () {58 vars in 1040 bytes} [Fri Jul 29 06:19:12 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:12.587424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfde59-63ca-4340-ab8c-939126598894 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:12.598993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfde59-63ca-4340-ab8c-939126598894 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:12.661690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 689/1401] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:12.705766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78027683-0aa5-4379-aa8a-d28c8f91d15b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:12.706331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78027683-0aa5-4379-aa8a-d28c8f91d15b service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:12.706444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:12.706609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78027683-0aa5-4379-aa8a-d28c8f91d15b service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:12.706883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78027683-0aa5-4379-aa8a-d28c8f91d15b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:12.707670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:12.709129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-78027683-0aa5-4379-aa8a-d28c8f91d15b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:12.736318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:12.797608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78027683-0aa5-4379-aa8a-d28c8f91d15b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:12.799200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78027683-0aa5-4379-aa8a-d28c8f91d15b service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:12.807339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 713/1402] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:19:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 06:19:12.831285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47404e0a-0183-4659-a8e8-3cbef4519931 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:12.953229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d7d7-1476-425a-bbe2-2232a613f535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:12.954308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d7d7-1476-425a-bbe2-2232a613f535 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:12.954681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d7d7-1476-425a-bbe2-2232a613f535 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:12.955024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d7d7-1476-425a-bbe2-2232a613f535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:12.956833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b720d7d7-1476-425a-bbe2-2232a613f535 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:12.978018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:13.066509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720d7d7-1476-425a-bbe2-2232a613f535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:13.069020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720d7d7-1476-425a-bbe2-2232a613f535 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:13.088357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 690/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:19:15.181597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d6781-efe9-46e6-8ba4-94776016b498 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:15.182221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d6781-efe9-46e6-8ba4-94776016b498 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:15.182221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d6781-efe9-46e6-8ba4-94776016b498 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:15.183601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 714/1404] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:15.189824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb366c-77f9-4ce7-87a2-27138d8a7677 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:15.190282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb366c-77f9-4ce7-87a2-27138d8a7677 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:15.190540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb366c-77f9-4ce7-87a2-27138d8a7677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:15.264462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5cbb366c-77f9-4ce7-87a2-27138d8a7677 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:15.265809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5cbb366c-77f9-4ce7-87a2-27138d8a7677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:15.373235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:15.380798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 691/1405] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:15.425818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d6781-efe9-46e6-8ba4-94776016b498 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:15.428236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-198d6781-efe9-46e6-8ba4-94776016b498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:15.439033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:15.595997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d6781-efe9-46e6-8ba4-94776016b498 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:15.688933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:15.727684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d9edc-234f-4d15-8842-5d550444b389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:15.728241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9edc-234f-4d15-8842-5d550444b389 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:15.728526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9edc-234f-4d15-8842-5d550444b389 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:15.728907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9edc-234f-4d15-8842-5d550444b389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:15.730719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7a8d9edc-234f-4d15-8842-5d550444b389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:15.832510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d9edc-234f-4d15-8842-5d550444b389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:15.834158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d9edc-234f-4d15-8842-5d550444b389 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:15.853662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 715/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:17.937044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea3d38-0746-4ada-a119-4483e0f26ddb None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:17.938018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea3d38-0746-4ada-a119-4483e0f26ddb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:17.938465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea3d38-0746-4ada-a119-4483e0f26ddb None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:17.941949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 692/1407] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:19:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:17.945980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264a90-5378-44fe-bdef-128b57f53573 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:17.946456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264a90-5378-44fe-bdef-128b57f53573 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:17.946559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264a90-5378-44fe-bdef-128b57f53573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:18.017545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fe264a90-5378-44fe-bdef-128b57f53573 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:18.018679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fe264a90-5378-44fe-bdef-128b57f53573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:18.073329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:18.144573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 716/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:18.179090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea3d38-0746-4ada-a119-4483e0f26ddb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:18.180648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-74ea3d38-0746-4ada-a119-4483e0f26ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:18.351379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea3d38-0746-4ada-a119-4483e0f26ddb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:18.425534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:18.477104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6084da-36ae-4b73-902e-7b388e32a33f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:18.477743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6084da-36ae-4b73-902e-7b388e32a33f service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:18.478083 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6084da-36ae-4b73-902e-7b388e32a33f service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:18.478364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6084da-36ae-4b73-902e-7b388e32a33f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:18.479989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5b6084da-36ae-4b73-902e-7b388e32a33f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:18.576579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6084da-36ae-4b73-902e-7b388e32a33f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:18.584912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6084da-36ae-4b73-902e-7b388e32a33f service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:18.602821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 693/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:28.127430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=23 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:28.600596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=64 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:35.355664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710c35e-96b3-4f99-afb5-236fbc7d4b62 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:35.356246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710c35e-96b3-4f99-afb5-236fbc7d4b62 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:35.356358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710c35e-96b3-4f99-afb5-236fbc7d4b62 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:35.357747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 717/1410] 10.222.0.33 () {54 vars in 918 bytes} [Fri Jul 29 06:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:35.362715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260c6db-861c-4056-b940-da295c6d29bb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:35.363145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260c6db-861c-4056-b940-da295c6d29bb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:35.363260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260c6db-861c-4056-b940-da295c6d29bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:35.440392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2260c6db-861c-4056-b940-da295c6d29bb None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:35.441620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2260c6db-861c-4056-b940-da295c6d29bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:35.547414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 694/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jul 29 06:19:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:35.562149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0b819-3a3a-47ba-bee2-03d50f33e3b7 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:35.562446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0b819-3a3a-47ba-bee2-03d50f33e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:35.562811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0b819-3a3a-47ba-bee2-03d50f33e3b7 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:35.564160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 718/1412] 10.222.0.33 () {54 vars in 918 bytes} [Fri Jul 29 06:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:35.569676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c810b-bfdb-4940-aa9c-0a72494de39a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:35.569966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c810b-bfdb-4940-aa9c-0a72494de39a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:35.570259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c810b-bfdb-4940-aa9c-0a72494de39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:35.643354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:35.646102 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9b7c810b-bfdb-4940-aa9c-0a72494de39a None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:35.647843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9b7c810b-bfdb-4940-aa9c-0a72494de39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:35.773731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 695/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jul 29 06:19:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:38.233936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b3d6c-2b63-444c-acce-9fb32fe3b246 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:38.234507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b3d6c-2b63-444c-acce-9fb32fe3b246 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:38.234507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b3d6c-2b63-444c-acce-9fb32fe3b246 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:38.235884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:38.241713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce151-07d5-43ee-88b7-eec29541cb04 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:38.242167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce151-07d5-43ee-88b7-eec29541cb04 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:38.242304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce151-07d5-43ee-88b7-eec29541cb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:38.308050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-cddce151-07d5-43ee-88b7-eec29541cb04 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:38.309258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-cddce151-07d5-43ee-88b7-eec29541cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:38.367413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:38.409336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:38.456153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b3d6c-2b63-444c-acce-9fb32fe3b246 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:38.458027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3b7b3d6c-2b63-444c-acce-9fb32fe3b246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:38.603302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b3d6c-2b63-444c-acce-9fb32fe3b246 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:38.675389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:38.716216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672a566-1364-47c0-9d93-a1396fecbb4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:38.716873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672a566-1364-47c0-9d93-a1396fecbb4d service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:38.717165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672a566-1364-47c0-9d93-a1396fecbb4d service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:38.717439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672a566-1364-47c0-9d93-a1396fecbb4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:38.719101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e672a566-1364-47c0-9d93-a1396fecbb4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:38.811798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672a566-1364-47c0-9d93-a1396fecbb4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:38.813429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672a566-1364-47c0-9d93-a1396fecbb4d service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:38.831919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 720/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:39.730864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7c66-1844-4684-93e0-2c9638b080b3 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:39.731429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7c66-1844-4684-93e0-2c9638b080b3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:39.731429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7c66-1844-4684-93e0-2c9638b080b3 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:39.732947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:19:39.737904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814fe29-d5cc-428e-ad95-19afe5416755 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:39.738313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814fe29-d5cc-428e-ad95-19afe5416755 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:39.738451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814fe29-d5cc-428e-ad95-19afe5416755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:39.813360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2814fe29-d5cc-428e-ad95-19afe5416755 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:39.814575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2814fe29-d5cc-428e-ad95-19afe5416755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:39.841952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:39.911768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:40.063685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7c66-1844-4684-93e0-2c9638b080b3 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:40.065199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-996b7c66-1844-4684-93e0-2c9638b080b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:40.212030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7c66-1844-4684-93e0-2c9638b080b3 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:40.252070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:40.323928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb997fe0-b807-49b9-911d-4d8c07171ab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:40.324492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997fe0-b807-49b9-911d-4d8c07171ab4 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:40.324804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997fe0-b807-49b9-911d-4d8c07171ab4 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:40.325072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997fe0-b807-49b9-911d-4d8c07171ab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:40.326668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-eb997fe0-b807-49b9-911d-4d8c07171ab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:40.416968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb997fe0-b807-49b9-911d-4d8c07171ab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:40.418563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb997fe0-b807-49b9-911d-4d8c07171ab4 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:40.436200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 698/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:41.316702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b476c-cd0f-4cb8-97dd-37463897de39 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:41.317927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b476c-cd0f-4cb8-97dd-37463897de39 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:41.317927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b476c-cd0f-4cb8-97dd-37463897de39 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:41.319363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:41.324191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779ff0b-bb5e-467b-b10b-8273ebf2e783 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:41.324615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779ff0b-bb5e-467b-b10b-8273ebf2e783 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:41.324788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779ff0b-bb5e-467b-b10b-8273ebf2e783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:41.378710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:41.400938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-0779ff0b-bb5e-467b-b10b-8273ebf2e783 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:41.402148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0779ff0b-bb5e-467b-b10b-8273ebf2e783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:41.500793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:41.653462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b476c-cd0f-4cb8-97dd-37463897de39 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:41.654977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-896b476c-cd0f-4cb8-97dd-37463897de39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:41.801913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b476c-cd0f-4cb8-97dd-37463897de39 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:41.813779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:41.913623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4935a6-e815-4a85-ba48-693a6988bb12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:41.914168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4935a6-e815-4a85-ba48-693a6988bb12 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:41.914452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4935a6-e815-4a85-ba48-693a6988bb12 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:41.914730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4935a6-e815-4a85-ba48-693a6988bb12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:41.916371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8c4935a6-e815-4a85-ba48-693a6988bb12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:42.003722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4935a6-e815-4a85-ba48-693a6988bb12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:42.005930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4935a6-e815-4a85-ba48-693a6988bb12 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:42.023567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 723/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:42.888947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca84b-38b7-41e6-ae07-c9bc6b4150ba None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:42.889513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca84b-38b7-41e6-ae07-c9bc6b4150ba None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:42.889513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca84b-38b7-41e6-ae07-c9bc6b4150ba None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:42.890842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:42.895922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47969-ab32-466d-8942-70df52deaac5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:42.896368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47969-ab32-466d-8942-70df52deaac5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:42.896552 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47969-ab32-466d-8942-70df52deaac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:42.916133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:42.972257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-19a47969-ab32-466d-8942-70df52deaac5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:42.976232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-19a47969-ab32-466d-8942-70df52deaac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:43.073056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:43.221537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca84b-38b7-41e6-ae07-c9bc6b4150ba None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:43.223049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-765ca84b-38b7-41e6-ae07-c9bc6b4150ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:43.332017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:43.368529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca84b-38b7-41e6-ae07-c9bc6b4150ba None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:43.484967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e1313-50b9-47b7-b37e-da76f26ef331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:43.485597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e1313-50b9-47b7-b37e-da76f26ef331 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:43.485888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e1313-50b9-47b7-b37e-da76f26ef331 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:43.486166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e1313-50b9-47b7-b37e-da76f26ef331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:43.487803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-524e1313-50b9-47b7-b37e-da76f26ef331 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:43.562907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:43.576626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e1313-50b9-47b7-b37e-da76f26ef331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:43.578217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e1313-50b9-47b7-b37e-da76f26ef331 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:43.596717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 701/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:44.456083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383d375-ccae-4e1b-ac64-55574f697ba3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:44.456791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383d375-ccae-4e1b-ac64-55574f697ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:44.456791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383d375-ccae-4e1b-ac64-55574f697ba3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:44.458704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:44.463574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199837e-ef3a-4827-874a-58fbffeca218 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:44.464024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199837e-ef3a-4827-874a-58fbffeca218 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:44.464135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199837e-ef3a-4827-874a-58fbffeca218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:44.539198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-6199837e-ef3a-4827-874a-58fbffeca218 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:44.540383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6199837e-ef3a-4827-874a-58fbffeca218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:44.637509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:44.787903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383d375-ccae-4e1b-ac64-55574f697ba3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:44.789449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7383d375-ccae-4e1b-ac64-55574f697ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:44.873726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:44.936324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383d375-ccae-4e1b-ac64-55574f697ba3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:45.046101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffee33-32a8-4b28-a28c-b9aeb1a2e74b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:45.046646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffee33-32a8-4b28-a28c-b9aeb1a2e74b service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:45.046926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffee33-32a8-4b28-a28c-b9aeb1a2e74b service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:45.047197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffee33-32a8-4b28-a28c-b9aeb1a2e74b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:45.048821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-58ffee33-32a8-4b28-a28c-b9aeb1a2e74b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:45.095441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:45.136980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffee33-32a8-4b28-a28c-b9aeb1a2e74b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:45.138576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffee33-32a8-4b28-a28c-b9aeb1a2e74b service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:45.156675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 726/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:45.977831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6362-f5e3-4f32-81dd-4d2e2aacf0b8 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:45.978442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6362-f5e3-4f32-81dd-4d2e2aacf0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:45.978442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6362-f5e3-4f32-81dd-4d2e2aacf0b8 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:45.979730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:45.984719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b85bd-cabf-4e28-b6fe-2db9115b48a7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:45.985133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b85bd-cabf-4e28-b6fe-2db9115b48a7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:45.985242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b85bd-cabf-4e28-b6fe-2db9115b48a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:46.061305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-114b85bd-cabf-4e28-b6fe-2db9115b48a7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:46.062504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-114b85bd-cabf-4e28-b6fe-2db9115b48a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:46.162400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:46.312901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca6362-f5e3-4f32-81dd-4d2e2aacf0b8 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:46.314377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-16ca6362-f5e3-4f32-81dd-4d2e2aacf0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:46.362618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:46.459395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca6362-f5e3-4f32-81dd-4d2e2aacf0b8 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:46.569955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60edbcc-a545-4646-991f-a0729e34da46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:46.570491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60edbcc-a545-4646-991f-a0729e34da46 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:46.570774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60edbcc-a545-4646-991f-a0729e34da46 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:46.571044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60edbcc-a545-4646-991f-a0729e34da46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:46.572670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b60edbcc-a545-4646-991f-a0729e34da46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:46.579688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:46.659820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60edbcc-a545-4646-991f-a0729e34da46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:46.661455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60edbcc-a545-4646-991f-a0729e34da46 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:46.679506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 704/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:47.484439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f82d-f90a-42cf-8b70-05d12d973e44 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:47.485040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f82d-f90a-42cf-8b70-05d12d973e44 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:47.485040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f82d-f90a-42cf-8b70-05d12d973e44 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:47.486379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:47.492461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc28c-7b44-4cf4-a0de-848c64a128c3 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:47.492968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc28c-7b44-4cf4-a0de-848c64a128c3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:47.493075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc28c-7b44-4cf4-a0de-848c64a128c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:47.569442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a4ffc28c-7b44-4cf4-a0de-848c64a128c3 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:47.570651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a4ffc28c-7b44-4cf4-a0de-848c64a128c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:47.637145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:47.666563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:47.818423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f82d-f90a-42cf-8b70-05d12d973e44 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:47.819928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3572f82d-f90a-42cf-8b70-05d12d973e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:47.837909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:47.967079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f82d-f90a-42cf-8b70-05d12d973e44 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:48.052272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:48.077920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c6a6c-0f38-4f8c-af6d-10f7e37029a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:48.078481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6a6c-0f38-4f8c-af6d-10f7e37029a1 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:48.078761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6a6c-0f38-4f8c-af6d-10f7e37029a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:48.079034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6a6c-0f38-4f8c-af6d-10f7e37029a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:48.080946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8a0c6a6c-0f38-4f8c-af6d-10f7e37029a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:48.169380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c6a6c-0f38-4f8c-af6d-10f7e37029a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:48.171044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c6a6c-0f38-4f8c-af6d-10f7e37029a1 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:48.189216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 729/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:19:48.994246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769cc36-7d75-494d-a0d6-9b88bf603127 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:48.994776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769cc36-7d75-494d-a0d6-9b88bf603127 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:48.994776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769cc36-7d75-494d-a0d6-9b88bf603127 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:48.996214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:49.001226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44aaf-ea44-4948-9b36-a3cbcad685f6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:49.001643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44aaf-ea44-4948-9b36-a3cbcad685f6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:49.001793 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44aaf-ea44-4948-9b36-a3cbcad685f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:49.076964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-44f44aaf-ea44-4948-9b36-a3cbcad685f6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:49.078204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-44f44aaf-ea44-4948-9b36-a3cbcad685f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:49.120200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:49.175483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:49.327765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769cc36-7d75-494d-a0d6-9b88bf603127 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:49.329593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c769cc36-7d75-494d-a0d6-9b88bf603127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:49.475162 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769cc36-7d75-494d-a0d6-9b88bf603127 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:49.529058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:49.588431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf954aa-0d5e-43ed-aef4-a9d01ac38945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:49.589089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf954aa-0d5e-43ed-aef4-a9d01ac38945 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:49.589373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf954aa-0d5e-43ed-aef4-a9d01ac38945 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:49.589648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf954aa-0d5e-43ed-aef4-a9d01ac38945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:49.591272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-daf954aa-0d5e-43ed-aef4-a9d01ac38945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:49.679538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf954aa-0d5e-43ed-aef4-a9d01ac38945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:49.681184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf954aa-0d5e-43ed-aef4-a9d01ac38945 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:49.699632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 707/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:50.510648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47104-841d-44e4-9a4d-d29a741c2cbe None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:50.511189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47104-841d-44e4-9a4d-d29a741c2cbe None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:50.511189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47104-841d-44e4-9a4d-d29a741c2cbe None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:50.512564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:50.517432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbef21-be51-48ae-b68b-cdc499542481 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:50.517861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbef21-be51-48ae-b68b-cdc499542481 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:50.517969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbef21-be51-48ae-b68b-cdc499542481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:50.594778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-31dbef21-be51-48ae-b68b-cdc499542481 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:50.595970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-31dbef21-be51-48ae-b68b-cdc499542481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:50.597988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:50.690144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:50.842738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47104-841d-44e4-9a4d-d29a741c2cbe None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:50.844259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3be47104-841d-44e4-9a4d-d29a741c2cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:50.991072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47104-841d-44e4-9a4d-d29a741c2cbe None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:51.017805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:51.104625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615622a-9e37-4da7-b85a-94de4dbf1a7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:51.105180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615622a-9e37-4da7-b85a-94de4dbf1a7f service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:51.105465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615622a-9e37-4da7-b85a-94de4dbf1a7f service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:51.105741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615622a-9e37-4da7-b85a-94de4dbf1a7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:51.107404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6615622a-9e37-4da7-b85a-94de4dbf1a7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:51.196352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615622a-9e37-4da7-b85a-94de4dbf1a7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:51.198003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615622a-9e37-4da7-b85a-94de4dbf1a7f service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:51.222925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 732/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:53.111010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1816-5a15-4d17-8cbe-d669d5735700 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:53.111604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1816-5a15-4d17-8cbe-d669d5735700 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:53.111604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1816-5a15-4d17-8cbe-d669d5735700 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:53.112943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:53.118808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ecedd-635b-46de-aed2-5a094dcf2028 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:53.119242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ecedd-635b-46de-aed2-5a094dcf2028 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:53.119398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ecedd-635b-46de-aed2-5a094dcf2028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:53.160153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:53.185825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ce3ecedd-635b-46de-aed2-5a094dcf2028 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:53.187023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ce3ecedd-635b-46de-aed2-5a094dcf2028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:53.282005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:53.320893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1816-5a15-4d17-8cbe-d669d5735700 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:53.322421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fbed1816-5a15-4d17-8cbe-d669d5735700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:53.423637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1816-5a15-4d17-8cbe-d669d5735700 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:53.456511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:53.498396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405bd45-5707-4fac-bb07-4df7c16f8619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:53.499022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405bd45-5707-4fac-bb07-4df7c16f8619 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:53.499305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405bd45-5707-4fac-bb07-4df7c16f8619 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:53.499579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405bd45-5707-4fac-bb07-4df7c16f8619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:53.501227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5405bd45-5707-4fac-bb07-4df7c16f8619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:53.608898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405bd45-5707-4fac-bb07-4df7c16f8619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:53.610327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405bd45-5707-4fac-bb07-4df7c16f8619 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:53.628965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 710/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:55.812926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff97cf-5881-4b08-8ad8-5dff5d0e2e72 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:55.813503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff97cf-5881-4b08-8ad8-5dff5d0e2e72 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:55.813503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff97cf-5881-4b08-8ad8-5dff5d0e2e72 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:55.814844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:55.820994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2dae-e16f-4c66-a07b-3c7437c261fc None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:55.821604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2dae-e16f-4c66-a07b-3c7437c261fc None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:55.821889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2dae-e16f-4c66-a07b-3c7437c261fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:55.905305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b71b2dae-e16f-4c66-a07b-3c7437c261fc None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:55.906515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b71b2dae-e16f-4c66-a07b-3c7437c261fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:55.919455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:56.010973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:19:56.048273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff97cf-5881-4b08-8ad8-5dff5d0e2e72 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:56.049842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cdff97cf-5881-4b08-8ad8-5dff5d0e2e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:56.154977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff97cf-5881-4b08-8ad8-5dff5d0e2e72 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:56.167138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:56.229645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0d53a-8741-4675-a6ba-f926c7da25be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:56.230260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0d53a-8741-4675-a6ba-f926c7da25be service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:56.230568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0d53a-8741-4675-a6ba-f926c7da25be service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:56.230843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0d53a-8741-4675-a6ba-f926c7da25be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:56.232479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a7b0d53a-8741-4675-a6ba-f926c7da25be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:56.341856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0d53a-8741-4675-a6ba-f926c7da25be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:56.343331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0d53a-8741-4675-a6ba-f926c7da25be service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:56.361343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 735/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:19:58.655165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3df88-59a2-4092-88dc-86ea0ed6e567 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:58.655731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3df88-59a2-4092-88dc-86ea0ed6e567 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:58.655731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3df88-59a2-4092-88dc-86ea0ed6e567 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:58.657140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:19:58.662418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314b447-5ee7-42d4-b139-b136a39f27d0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:58.662848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314b447-5ee7-42d4-b139-b136a39f27d0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:58.662960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314b447-5ee7-42d4-b139-b136a39f27d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:58.712039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:58.731147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7314b447-5ee7-42d4-b139-b136a39f27d0 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:19:58.732510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7314b447-5ee7-42d4-b139-b136a39f27d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:58.828146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:19:58.867762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3df88-59a2-4092-88dc-86ea0ed6e567 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:19:58.869334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f9c3df88-59a2-4092-88dc-86ea0ed6e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:58.972107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3df88-59a2-4092-88dc-86ea0ed6e567 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:19:59.012767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:19:59.046878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbfce-d424-4d82-80f7-19a218c8c89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:19:59.047511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbfce-d424-4d82-80f7-19a218c8c89c service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:19:59.047792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbfce-d424-4d82-80f7-19a218c8c89c service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:19:59.048067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbfce-d424-4d82-80f7-19a218c8c89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:19:59.049723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-07fdbfce-d424-4d82-80f7-19a218c8c89c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:19:59.158699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdbfce-d424-4d82-80f7-19a218c8c89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:19:59.160133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdbfce-d424-4d82-80f7-19a218c8c89c service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:19:59.179502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 713/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:01.297738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea62098-b27e-4e9c-bad8-6a08a6c7f5ee None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:01.298304 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea62098-b27e-4e9c-bad8-6a08a6c7f5ee None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:01.298405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea62098-b27e-4e9c-bad8-6a08a6c7f5ee None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:01.299734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:01.306052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b699-edd9-4b63-9608-462968eccfa8 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:01.306472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b699-edd9-4b63-9608-462968eccfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:01.306613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b699-edd9-4b63-9608-462968eccfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:01.372006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e790b699-edd9-4b63-9608-462968eccfa8 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:01.373228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e790b699-edd9-4b63-9608-462968eccfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:01.393270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:01.471910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:01.509875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea62098-b27e-4e9c-bad8-6a08a6c7f5ee None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:01.511420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0ea62098-b27e-4e9c-bad8-6a08a6c7f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:01.618268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea62098-b27e-4e9c-bad8-6a08a6c7f5ee None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:01.638484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:01.691529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4da3c-833c-43c9-9c33-295e23f458ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:01.692721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4da3c-833c-43c9-9c33-295e23f458ca service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:01.693005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4da3c-833c-43c9-9c33-295e23f458ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:01.693279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4da3c-833c-43c9-9c33-295e23f458ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:01.694876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ecf4da3c-833c-43c9-9c33-295e23f458ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:01.807728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4da3c-833c-43c9-9c33-295e23f458ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:01.809241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4da3c-833c-43c9-9c33-295e23f458ca service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:01.827057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 738/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:03.884080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c9162-a642-4fb2-96f3-f4e685581e79 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:03.884648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c9162-a642-4fb2-96f3-f4e685581e79 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:03.884648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c9162-a642-4fb2-96f3-f4e685581e79 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:03.886052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:03.891200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190f743-2b7f-4443-98d3-ab0b11bb522b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:03.891655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190f743-2b7f-4443-98d3-ab0b11bb522b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:03.891753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190f743-2b7f-4443-98d3-ab0b11bb522b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:04.080601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:04.083584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b190f743-2b7f-4443-98d3-ab0b11bb522b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:04.085409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b190f743-2b7f-4443-98d3-ab0b11bb522b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:04.182753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:04.220020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c9162-a642-4fb2-96f3-f4e685581e79 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:04.221609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8e5c9162-a642-4fb2-96f3-f4e685581e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:04.323601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c9162-a642-4fb2-96f3-f4e685581e79 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:04.373869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:04.399198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58528c38-6416-4ad9-9b98-924443330afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:04.399822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58528c38-6416-4ad9-9b98-924443330afe service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:04.400107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58528c38-6416-4ad9-9b98-924443330afe service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:04.400385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58528c38-6416-4ad9-9b98-924443330afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:04.402041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-58528c38-6416-4ad9-9b98-924443330afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:04.511858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58528c38-6416-4ad9-9b98-924443330afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:04.513348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58528c38-6416-4ad9-9b98-924443330afe service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:04.532110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 716/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:06.628865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03802c8-818d-4d32-b9b5-a59703609205 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:06.630006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03802c8-818d-4d32-b9b5-a59703609205 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:06.630006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03802c8-818d-4d32-b9b5-a59703609205 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:06.631537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:06.637038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ebfd6-9484-413f-b8ee-276bf4f8fd76 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:06.637466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ebfd6-9484-413f-b8ee-276bf4f8fd76 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:06.637596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ebfd6-9484-413f-b8ee-276bf4f8fd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:06.703203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e41ebfd6-9484-413f-b8ee-276bf4f8fd76 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:06.704424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e41ebfd6-9484-413f-b8ee-276bf4f8fd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:06.734506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:06.804430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:06.842427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03802c8-818d-4d32-b9b5-a59703609205 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:06.843992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d03802c8-818d-4d32-b9b5-a59703609205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:06.946807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03802c8-818d-4d32-b9b5-a59703609205 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:06.974987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:07.020429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040b54c-0667-4981-931e-2a83903989d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:07.021078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b54c-0667-4981-931e-2a83903989d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:07.021359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b54c-0667-4981-931e-2a83903989d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:07.021635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b54c-0667-4981-931e-2a83903989d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:07.023238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3040b54c-0667-4981-931e-2a83903989d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:07.133153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040b54c-0667-4981-931e-2a83903989d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:07.134587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040b54c-0667-4981-931e-2a83903989d0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:07.152943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 741/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:09.230261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f01580-9c6a-4950-84f5-f5d42d379d42 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:09.230784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f01580-9c6a-4950-84f5-f5d42d379d42 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:09.230880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f01580-9c6a-4950-84f5-f5d42d379d42 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:09.232197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:09.237436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e4e6e-c5f4-44c1-8b95-79526faaf37c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:09.237928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e4e6e-c5f4-44c1-8b95-79526faaf37c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:09.238036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e4e6e-c5f4-44c1-8b95-79526faaf37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:09.304314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-df1e4e6e-c5f4-44c1-8b95-79526faaf37c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:09.305550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-df1e4e6e-c5f4-44c1-8b95-79526faaf37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:09.313650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:09.401289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:09.439258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f01580-9c6a-4950-84f5-f5d42d379d42 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:09.440906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-97f01580-9c6a-4950-84f5-f5d42d379d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:09.543589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f01580-9c6a-4950-84f5-f5d42d379d42 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:09.597690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:09.617658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c2e38-691b-4cce-a4a7-c98ad811c546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:09.618280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c2e38-691b-4cce-a4a7-c98ad811c546 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:09.618562 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c2e38-691b-4cce-a4a7-c98ad811c546 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:09.618838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c2e38-691b-4cce-a4a7-c98ad811c546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:09.620446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ac7c2e38-691b-4cce-a4a7-c98ad811c546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:09.729140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c2e38-691b-4cce-a4a7-c98ad811c546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:09.730572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c2e38-691b-4cce-a4a7-c98ad811c546 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:09.749563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 719/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:11.792059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac669f-3bd0-42cf-be62-d654f23f7d11 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:11.792613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac669f-3bd0-42cf-be62-d654f23f7d11 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:11.792710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac669f-3bd0-42cf-be62-d654f23f7d11 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:11.794010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:11.799260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bf155-e403-4e80-ad3f-2acfab5f8860 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:11.799683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bf155-e403-4e80-ad3f-2acfab5f8860 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:11.799798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bf155-e403-4e80-ad3f-2acfab5f8860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:11.865711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a46bf155-e403-4e80-ad3f-2acfab5f8860 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:11.866959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a46bf155-e403-4e80-ad3f-2acfab5f8860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:11.905224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:11.964392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:12.002383 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac669f-3bd0-42cf-be62-d654f23f7d11 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:12.003995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8cac669f-3bd0-42cf-be62-d654f23f7d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:12.107374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac669f-3bd0-42cf-be62-d654f23f7d11 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:12.143945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:12.180599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a3d7c-4e12-447c-ad18-3fcda4b795b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:12.181227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3d7c-4e12-447c-ad18-3fcda4b795b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:12.181509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3d7c-4e12-447c-ad18-3fcda4b795b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:12.181788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3d7c-4e12-447c-ad18-3fcda4b795b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:12.183394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-262a3d7c-4e12-447c-ad18-3fcda4b795b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:12.290345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a3d7c-4e12-447c-ad18-3fcda4b795b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:12.291707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a3d7c-4e12-447c-ad18-3fcda4b795b1 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:12.309239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 744/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:14.330318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58d12-a284-41e6-b0e0-3457de679679 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:14.330896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58d12-a284-41e6-b0e0-3457de679679 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:14.330896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58d12-a284-41e6-b0e0-3457de679679 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:14.332456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:14.337604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd9692-0a25-47c0-9018-77c4d4da51c9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:14.338012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd9692-0a25-47c0-9018-77c4d4da51c9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:14.338165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd9692-0a25-47c0-9018-77c4d4da51c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:14.404060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-60cd9692-0a25-47c0-9018-77c4d4da51c9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:14.405239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-60cd9692-0a25-47c0-9018-77c4d4da51c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:14.423277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:14.503340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:14.540334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a58d12-a284-41e6-b0e0-3457de679679 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:14.541928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e6a58d12-a284-41e6-b0e0-3457de679679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:14.647443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a58d12-a284-41e6-b0e0-3457de679679 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:14.713073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:14.720516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d26e9c-0ca9-4dfc-85e8-caabbcd623d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:14.721527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d26e9c-0ca9-4dfc-85e8-caabbcd623d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:14.721809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d26e9c-0ca9-4dfc-85e8-caabbcd623d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:14.722085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d26e9c-0ca9-4dfc-85e8-caabbcd623d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:14.724219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d1d26e9c-0ca9-4dfc-85e8-caabbcd623d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:14.832626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d26e9c-0ca9-4dfc-85e8-caabbcd623d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:14.834048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d26e9c-0ca9-4dfc-85e8-caabbcd623d5 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:14.853313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 722/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:16.853418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cbd15-c996-4378-9550-33fca006ee71 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:16.854017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cbd15-c996-4378-9550-33fca006ee71 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:16.854017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cbd15-c996-4378-9550-33fca006ee71 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:16.855338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:16.860456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170c60-4793-429a-8ace-4594b73d2d3b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:16.860906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170c60-4793-429a-8ace-4594b73d2d3b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:16.861056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170c60-4793-429a-8ace-4594b73d2d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:16.927062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-20170c60-4793-429a-8ace-4594b73d2d3b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:16.928253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-20170c60-4793-429a-8ace-4594b73d2d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:16.976560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:17.025657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:17.061945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cbd15-c996-4378-9550-33fca006ee71 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:17.063487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ac1cbd15-c996-4378-9550-33fca006ee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:17.164945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cbd15-c996-4378-9550-33fca006ee71 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:17.212107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:17.237267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f077615-1f6a-4703-ad17-02a3ea9b213e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:17.237884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f077615-1f6a-4703-ad17-02a3ea9b213e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:17.238162 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f077615-1f6a-4703-ad17-02a3ea9b213e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:17.239028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f077615-1f6a-4703-ad17-02a3ea9b213e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:17.240674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8f077615-1f6a-4703-ad17-02a3ea9b213e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:17.346710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f077615-1f6a-4703-ad17-02a3ea9b213e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:17.348206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f077615-1f6a-4703-ad17-02a3ea9b213e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:17.366401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 747/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:27.023807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=22 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:27.363534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=66 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:37.718028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800def6-1dac-4061-963b-bd145906effe None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:37.718620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800def6-1dac-4061-963b-bd145906effe None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:37.718620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800def6-1dac-4061-963b-bd145906effe None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:37.720291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 724/1471] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:37.725400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c3401-f521-4b77-8a0b-d66dbc88674b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:37.725865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c3401-f521-4b77-8a0b-d66dbc88674b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:37.725966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c3401-f521-4b77-8a0b-d66dbc88674b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:37.791853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c05c3401-f521-4b77-8a0b-d66dbc88674b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:37.793079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c05c3401-f521-4b77-8a0b-d66dbc88674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:37.890156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 748/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:37.974366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9868d6c-c57e-42f9-8267-be1d8c1db92d None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:37.974658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9868d6c-c57e-42f9-8267-be1d8c1db92d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:37.974941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9868d6c-c57e-42f9-8267-be1d8c1db92d None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:37.976319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 725/1473] 10.222.0.33 () {54 vars in 966 bytes} [Fri Jul 29 06:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:37.981223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e448af1-ce1e-4c4a-b894-66947649e8a5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:37.981515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e448af1-ce1e-4c4a-b894-66947649e8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:37.981794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e448af1-ce1e-4c4a-b894-66947649e8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:38.055966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4e448af1-ce1e-4c4a-b894-66947649e8a5 None None] MFA Rules not processed for user `bc4b8f7545934151aafa986da871b944`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:38.057193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4e448af1-ce1e-4c4a-b894-66947649e8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:38.078264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:38.136054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 749/1474] 10.222.0.33 () {58 vars in 1063 bytes} [Fri Jul 29 06:20:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:38.143751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9868d6c-c57e-42f9-8267-be1d8c1db92d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:38.145432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b9868d6c-c57e-42f9-8267-be1d8c1db92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:38.249175 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9868d6c-c57e-42f9-8267-be1d8c1db92d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:38.322476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcbbb1-cb3c-42c2-a712-15fecaa50bab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4b8f7545934151aafa986da871b944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:38.323094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbbb1-cb3c-42c2-a712-15fecaa50bab service glance] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:38.323373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbbb1-cb3c-42c2-a712-15fecaa50bab service glance] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:38.323647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbbb1-cb3c-42c2-a712-15fecaa50bab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:38.325274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-75fcbbb1-cb3c-42c2-a712-15fecaa50bab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:38.380189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:38.434897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcbbb1-cb3c-42c2-a712-15fecaa50bab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:38.436325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcbbb1-cb3c-42c2-a712-15fecaa50bab service glance] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:38.456273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 726/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:40.537632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0fcd5-79f0-40f3-845d-5537b81ed1d4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:40.538186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0fcd5-79f0-40f3-845d-5537b81ed1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:40.538186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0fcd5-79f0-40f3-845d-5537b81ed1d4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:40.539450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 750/1476] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:40.544588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a02c60-c974-45f4-a2fd-2ab499c9bfc6 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:40.545039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a02c60-c974-45f4-a2fd-2ab499c9bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:40.545144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a02c60-c974-45f4-a2fd-2ab499c9bfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:40.610932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-53a02c60-c974-45f4-a2fd-2ab499c9bfc6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:40.612153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-53a02c60-c974-45f4-a2fd-2ab499c9bfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:40.713784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 727/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:40.737649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0fcd5-79f0-40f3-845d-5537b81ed1d4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:40.739755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eeb0fcd5-79f0-40f3-845d-5537b81ed1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:40.863941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0fcd5-79f0-40f3-845d-5537b81ed1d4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:40.912834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:40.958717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd2e74-4124-40ce-bbaf-20bc00957795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:40.959375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2e74-4124-40ce-bbaf-20bc00957795 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:40.959692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2e74-4124-40ce-bbaf-20bc00957795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:40.959992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2e74-4124-40ce-bbaf-20bc00957795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:40.960741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd2e74-4124-40ce-bbaf-20bc00957795 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:40.962501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-fedd2e74-4124-40ce-bbaf-20bc00957795 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:20:40.962501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:20:40.962501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:20:40.963641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd2e74-4124-40ce-bbaf-20bc00957795 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:40.982212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 751/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 29 06:20:40 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:20:43.023833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fce966-07ff-4176-84b0-b9e30ee42281 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:43.024364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fce966-07ff-4176-84b0-b9e30ee42281 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:43.024364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fce966-07ff-4176-84b0-b9e30ee42281 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:43.025669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 728/1479] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:43.030915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4675d-7e00-411c-a46a-87c0f1b4427a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:43.031331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4675d-7e00-411c-a46a-87c0f1b4427a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:43.031449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4675d-7e00-411c-a46a-87c0f1b4427a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:43.097168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d3c4675d-7e00-411c-a46a-87c0f1b4427a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:43.098372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d3c4675d-7e00-411c-a46a-87c0f1b4427a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:43.194092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 752/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:43.236737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fce966-07ff-4176-84b0-b9e30ee42281 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:43.238876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e3fce966-07ff-4176-84b0-b9e30ee42281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:43.248852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:43.386022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fce966-07ff-4176-84b0-b9e30ee42281 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:43.466691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:43.500438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989579f-fd1d-4ad1-a382-d059fd90e825 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:43.501097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989579f-fd1d-4ad1-a382-d059fd90e825 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:43.501383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989579f-fd1d-4ad1-a382-d059fd90e825 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:43.501663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989579f-fd1d-4ad1-a382-d059fd90e825 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:43.503283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0989579f-fd1d-4ad1-a382-d059fd90e825 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:43.591918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989579f-fd1d-4ad1-a382-d059fd90e825 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:43.593520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989579f-fd1d-4ad1-a382-d059fd90e825 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:43.611632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 729/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:45.793169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbef7d-19a7-4612-bba9-5b04d487e6bd None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:45.793694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbef7d-19a7-4612-bba9-5b04d487e6bd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:45.793694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbef7d-19a7-4612-bba9-5b04d487e6bd None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:45.795096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 753/1482] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:45.800467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b864840-dbe8-45be-940f-0b3b781a4d1d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:45.800918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b864840-dbe8-45be-940f-0b3b781a4d1d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:45.801079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b864840-dbe8-45be-940f-0b3b781a4d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:45.866927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-0b864840-dbe8-45be-940f-0b3b781a4d1d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:45.868134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0b864840-dbe8-45be-940f-0b3b781a4d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:45.913998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:45.964185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 730/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:46.000657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbef7d-19a7-4612-bba9-5b04d487e6bd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:46.002219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9afbef7d-19a7-4612-bba9-5b04d487e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:46.025620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:46.107040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbef7d-19a7-4612-bba9-5b04d487e6bd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:46.187392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e69e56-c6fe-4f5d-924c-b853f63e4801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:46.188017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e69e56-c6fe-4f5d-924c-b853f63e4801 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:46.188300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e69e56-c6fe-4f5d-924c-b853f63e4801 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:46.188600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e69e56-c6fe-4f5d-924c-b853f63e4801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:46.190220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-97e69e56-c6fe-4f5d-924c-b853f63e4801 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:46.264629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:46.303347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e69e56-c6fe-4f5d-924c-b853f63e4801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:46.304826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e69e56-c6fe-4f5d-924c-b853f63e4801 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:46.322780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 754/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:48.368837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436d44-bb1d-4faf-8bf4-b8ee3b814bd8 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:48.369440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436d44-bb1d-4faf-8bf4-b8ee3b814bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:48.369440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436d44-bb1d-4faf-8bf4-b8ee3b814bd8 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:48.370750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 731/1485] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:48.375871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88de52a-7ed0-463e-8425-b47fc4f6c10e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:48.376315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88de52a-7ed0-463e-8425-b47fc4f6c10e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:48.376422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88de52a-7ed0-463e-8425-b47fc4f6c10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:48.440898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d88de52a-7ed0-463e-8425-b47fc4f6c10e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:48.442230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d88de52a-7ed0-463e-8425-b47fc4f6c10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:48.545833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 755/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:48.585260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f436d44-bb1d-4faf-8bf4-b8ee3b814bd8 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:48.586856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6f436d44-bb1d-4faf-8bf4-b8ee3b814bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:48.690317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f436d44-bb1d-4faf-8bf4-b8ee3b814bd8 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:48.762662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:48.764604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb52fb4-c62b-4450-97a5-7989c705a428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:48.765251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52fb4-c62b-4450-97a5-7989c705a428 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:48.765729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52fb4-c62b-4450-97a5-7989c705a428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:48.766227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52fb4-c62b-4450-97a5-7989c705a428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:48.768073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-afb52fb4-c62b-4450-97a5-7989c705a428 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:48.875679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb52fb4-c62b-4450-97a5-7989c705a428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:48.877146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb52fb4-c62b-4450-97a5-7989c705a428 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:48.896438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 732/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:50.982722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f16603-dcfb-4ba2-9a24-da60e51e7d94 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:50.983246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f16603-dcfb-4ba2-9a24-da60e51e7d94 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:50.983246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f16603-dcfb-4ba2-9a24-da60e51e7d94 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:50.984602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 756/1488] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:50.989901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c7c0-51c8-4e10-863d-de00a20df2ef None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:50.990348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c7c0-51c8-4e10-863d-de00a20df2ef None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:50.990445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c7c0-51c8-4e10-863d-de00a20df2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:51.055442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-bb83c7c0-51c8-4e10-863d-de00a20df2ef None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:51.056671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bb83c7c0-51c8-4e10-863d-de00a20df2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:51.108946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:51.152630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 733/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:20:51.193243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f16603-dcfb-4ba2-9a24-da60e51e7d94 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:51.194775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-01f16603-dcfb-4ba2-9a24-da60e51e7d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:51.230929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:51.299306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f16603-dcfb-4ba2-9a24-da60e51e7d94 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:51.372314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a745ab-6fb4-4eea-b795-7b4ce448928c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:51.372960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a745ab-6fb4-4eea-b795-7b4ce448928c service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:51.373238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a745ab-6fb4-4eea-b795-7b4ce448928c service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:51.373511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a745ab-6fb4-4eea-b795-7b4ce448928c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:51.376200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-70a745ab-6fb4-4eea-b795-7b4ce448928c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:51.458843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:51.489027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a745ab-6fb4-4eea-b795-7b4ce448928c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:51.490475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a745ab-6fb4-4eea-b795-7b4ce448928c service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:51.508819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 757/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:20:53.791561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f09e3-0ed5-4171-adbf-cdd74fde5dde None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:53.792090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f09e3-0ed5-4171-adbf-cdd74fde5dde None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:53.792205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f09e3-0ed5-4171-adbf-cdd74fde5dde None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:53.793550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 734/1491] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 29 06:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:20:53.798890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f94342-22c8-45c2-9708-fa92d224ad7a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:53.799281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f94342-22c8-45c2-9708-fa92d224ad7a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:53.799457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f94342-22c8-45c2-9708-fa92d224ad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:53.868489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-96f94342-22c8-45c2-9708-fa92d224ad7a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:20:53.869727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-96f94342-22c8-45c2-9708-fa92d224ad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:53.966767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 758/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 29 06:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:20:54.005888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f09e3-0ed5-4171-adbf-cdd74fde5dde None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:20:54.007455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c49f09e3-0ed5-4171-adbf-cdd74fde5dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:54.111180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f09e3-0ed5-4171-adbf-cdd74fde5dde None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:20:54.184672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96950eb-d246-44d8-bf29-adc30984bc1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:20:54.185297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96950eb-d246-44d8-bf29-adc30984bc1e service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:20:54.185583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96950eb-d246-44d8-bf29-adc30984bc1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:20:54.185860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96950eb-d246-44d8-bf29-adc30984bc1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:20:54.187477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e96950eb-d246-44d8-bf29-adc30984bc1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:20:54.207035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:20:54.302647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96950eb-d246-44d8-bf29-adc30984bc1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:20:54.303997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96950eb-d246-44d8-bf29-adc30984bc1e service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:20:54.323045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 735/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:21:03.964849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=82 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:21:04.320764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=50 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:43.467485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:43.468133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:43.468133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:43.544888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:22:43.546099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:43.641315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 759/1494] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:22:43.650554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb281-bb1b-42d1-837a-78a1e97efe52 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:43.651076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb281-bb1b-42d1-837a-78a1e97efe52 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:43.651203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb281-bb1b-42d1-837a-78a1e97efe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:43.726317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-57bdb281-bb1b-42d1-837a-78a1e97efe52 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:22:43.727440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-57bdb281-bb1b-42d1-837a-78a1e97efe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:43.822478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 736/1495] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:22:43.827135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:43.828327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:43.840651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a089ea-e219-405d-b2c8-14a41f6eb8bd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:43.901929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:43.935125 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:43.935741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:43.936018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:43.936314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:43.937023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:22:43.937327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:43.938331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:43.938331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:22:43.938331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:22:43.939264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:43.950948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 760/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:22:43.959862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb281-bb1b-42d1-837a-78a1e97efe52 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:43.969027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb281-bb1b-42d1-837a-78a1e97efe52 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:44.029722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:44.063924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:44.064575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.064862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.065140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:44.065812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:44.067521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING py.warnings [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:44.067521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: warnings.warn(msg) Jul 29 06:22:44.067521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: Jul 29 06:22:44.068771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:44.124845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 737/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:22:43 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:22:44.129077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:44.130616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:44.164366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab1f18-6d6d-4589-b133-50cf8ebd36ec admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:44.257899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:44.258520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.258800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.259072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:44.259727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:44.260757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:44.260757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:22:44.260757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:22:44.261947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:44.303375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 761/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:22:44 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:22:44.307321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:44.308874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:44.370915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b37a7-549f-44f3-b875-aebb89562613 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:44.454920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:44.467964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:44.468664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.469006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.469315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:44.470122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:44.471235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING py.warnings [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:44.471235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: warnings.warn(msg) Jul 29 06:22:44.471235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: Jul 29 06:22:44.472110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:44.485245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 738/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:22:44.489320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:44.491520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:44.502812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078a06-70ef-45ee-9151-6e99a40175ba admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:44.527399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:44.596603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dba26-c5ea-4f60-91a0-5d47a7c85efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:44.597220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dba26-c5ea-4f60-91a0-5d47a7c85efc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.597500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dba26-c5ea-4f60-91a0-5d47a7c85efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.597771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dba26-c5ea-4f60-91a0-5d47a7c85efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:44.598452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dba26-c5ea-4f60-91a0-5d47a7c85efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:44.599451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-616dba26-c5ea-4f60-91a0-5d47a7c85efc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:44.599451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:22:44.599451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:22:44.600901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dba26-c5ea-4f60-91a0-5d47a7c85efc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:44.613183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 762/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:22:44.618159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:44.619721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:44.652305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb92ea3-f4b8-4ecb-8e23-aa61e583400c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:44.744779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576fec9-1ada-4221-be8e-4db2f2fe827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:44.745457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576fec9-1ada-4221-be8e-4db2f2fe827d admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.745733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576fec9-1ada-4221-be8e-4db2f2fe827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.746002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576fec9-1ada-4221-be8e-4db2f2fe827d admin admin] PATH_INFO: `/v3/projects/c8d6ba4774a142fd9bde6c1c107e9066/users/1e3cb3d92efd43f39206cd6048152ceb/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:44.768368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576fec9-1ada-4221-be8e-4db2f2fe827d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d6ba4774a142fd9bde6c1c107e9066, user_id=1e3cb3d92efd43f39206cd6048152ceb, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:44.769425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING py.warnings [None req-7576fec9-1ada-4221-be8e-4db2f2fe827d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:44.769425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: warnings.warn(msg) Jul 29 06:22:44.769425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: Jul 29 06:22:44.771350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576fec9-1ada-4221-be8e-4db2f2fe827d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:44.796409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 739/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:22:44 2022] PUT /identity/v3/projects/c8d6ba4774a142fd9bde6c1c107e9066/users/1e3cb3d92efd43f39206cd6048152ceb/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:22:44.809141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e51e8-19ff-4591-b4d3-d593f73c6967 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.809430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e51e8-19ff-4591-b4d3-d593f73c6967 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.809706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e51e8-19ff-4591-b4d3-d593f73c6967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:44.867386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f93e51e8-19ff-4591-b4d3-d593f73c6967 None None] MFA Rules not processed for user `1e3cb3d92efd43f39206cd6048152ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:22:44.869421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f93e51e8-19ff-4591-b4d3-d593f73c6967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:44.950606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 763/1502] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:22:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:22:44.964979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26187763-9eda-49d8-8883-360fc963b03b None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.965274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26187763-9eda-49d8-8883-360fc963b03b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.965551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26187763-9eda-49d8-8883-360fc963b03b None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:44.966914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 740/1503] 10.222.0.33 () {54 vars in 964 bytes} [Fri Jul 29 06:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:22:44.971700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f8af-31c0-45a1-8210-6bfc1d8ebb4a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:44.971990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f8af-31c0-45a1-8210-6bfc1d8ebb4a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:44.972267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f8af-31c0-45a1-8210-6bfc1d8ebb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:45.002950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:45.047099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4253f8af-31c0-45a1-8210-6bfc1d8ebb4a None None] MFA Rules not processed for user `ec1feaf015a847ffa719fd3a53de02e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:22:45.048287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4253f8af-31c0-45a1-8210-6bfc1d8ebb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:45.122332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 764/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Fri Jul 29 06:22:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:22:45.130112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26187763-9eda-49d8-8883-360fc963b03b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:45.132157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-26187763-9eda-49d8-8883-360fc963b03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:45.182935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:45.236163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26187763-9eda-49d8-8883-360fc963b03b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:45.307872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:45.308488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:45.308796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:45.309072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:45.311060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:45.415506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:45.416887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:45.420333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:45.436343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 741/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:22:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:22:45.486280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee65c-92e9-4ab9-8554-39396094a546 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:45.486575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee65c-92e9-4ab9-8554-39396094a546 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:45.486861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee65c-92e9-4ab9-8554-39396094a546 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:45.488136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 765/1506] 10.222.0.33 () {52 vars in 830 bytes} [Fri Jul 29 06:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 06:22:45.492362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:45.493632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:45.591244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d050a0-298f-4af9-8d6f-15682263214a service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:45.683097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:45.683713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:45.683936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:45.684210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] PATH_INFO: `/v3/users/1e3cb3d92efd43f39206cd6048152ceb` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:45.696852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3cb3d92efd43f39206cd6048152ceb)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:45.697868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING py.warnings [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:45.697868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: warnings.warn(msg) Jul 29 06:22:45.697868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: Jul 29 06:22:45.699071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:45.708935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:45.763972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3cb3d92efd43f39206cd6048152ceb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:45.764416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3cb3d92efd43f39206cd6048152ceb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:45.782381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3cb3d92efd43f39206cd6048152ceb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:45.794477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3cb3d92efd43f39206cd6048152ceb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:45.795025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3cb3d92efd43f39206cd6048152ceb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:45.810373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 742/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:22:45 2022] DELETE /identity/v3/users/1e3cb3d92efd43f39206cd6048152ceb => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:22:45.814834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:45.815311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:45.815407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:45.872704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:22:45.873918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:45.949703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:45.963452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 766/1508] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:22:45.973161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:45.975001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:46.072461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4537e2-508e-41c0-a9ac-759df1c1fa8f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:46.128738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:46.149220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228e3ee-5a30-4cc7-b8f8-46fd66dd0af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:46.149855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228e3ee-5a30-4cc7-b8f8-46fd66dd0af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:46.150138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228e3ee-5a30-4cc7-b8f8-46fd66dd0af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:46.150417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228e3ee-5a30-4cc7-b8f8-46fd66dd0af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:46.152068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3228e3ee-5a30-4cc7-b8f8-46fd66dd0af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:46.261639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228e3ee-5a30-4cc7-b8f8-46fd66dd0af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:46.262993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228e3ee-5a30-4cc7-b8f8-46fd66dd0af1 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:46.280930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 743/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:22:46.624901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:22:46.626133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:22:46.638327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20bb91-7e40-4528-ba30-433acdb4adc7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:22:46.748378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:22:46.749053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:22:46.749336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:22:46.749612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] PATH_INFO: `/v3/projects/c8d6ba4774a142fd9bde6c1c107e9066` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:22:46.759977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d6ba4774a142fd9bde6c1c107e9066)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:22:46.761067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:22:46.761067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:22:46.761067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:22:46.762259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:22:46.818935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d6ba4774a142fd9bde6c1c107e9066', 'request_id': 'req-d2d75cba-1e8a-4c1b-9b77-fe569165177e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:46.827877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d6ba4774a142fd9bde6c1c107e9066', 'request_id': 'req-d2d75cba-1e8a-4c1b-9b77-fe569165177e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:46.842013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d6ba4774a142fd9bde6c1c107e9066', 'request_id': 'req-d2d75cba-1e8a-4c1b-9b77-fe569165177e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:46.864146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] The token cache is being invalidate because project c8d6ba4774a142fd9bde6c1c107e9066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:22:46.864562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:46.865000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d2d75cba-1e8a-4c1b-9b77-fe569165177e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:22:46.866721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 767/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:22:46 2022] DELETE /identity/v3/projects/c8d6ba4774a142fd9bde6c1c107e9066 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:22:56.278614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=62 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:22:56.862180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:24:25.529422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aedcd-1339-40b8-b0a1-2bedf35d1d20 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:25.530120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aedcd-1339-40b8-b0a1-2bedf35d1d20 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:25.530120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aedcd-1339-40b8-b0a1-2bedf35d1d20 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:25.531376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:25.537024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9e3b-fdb5-45d3-90c4-363031d9b109 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:25.537477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9e3b-fdb5-45d3-90c4-363031d9b109 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:25.537587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9e3b-fdb5-45d3-90c4-363031d9b109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:25.617951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-effb9e3b-fdb5-45d3-90c4-363031d9b109 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:25.619158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-effb9e3b-fdb5-45d3-90c4-363031d9b109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:25.721522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:25.882587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aedcd-1339-40b8-b0a1-2bedf35d1d20 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:25.884714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bf0aedcd-1339-40b8-b0a1-2bedf35d1d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:26.011560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aedcd-1339-40b8-b0a1-2bedf35d1d20 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:26.017244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963cc0a-66de-48ce-b061-a0676685d6b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:26.017882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963cc0a-66de-48ce-b061-a0676685d6b7 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:26.018173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963cc0a-66de-48ce-b061-a0676685d6b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:26.018458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963cc0a-66de-48ce-b061-a0676685d6b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:26.027517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963cc0a-66de-48ce-b061-a0676685d6b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:26.029147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963cc0a-66de-48ce-b061-a0676685d6b7 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:26.039452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 745/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:26.871537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9aa8b-12a8-4df5-a934-ac516bc6cc3d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:26.872138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9aa8b-12a8-4df5-a934-ac516bc6cc3d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:26.872138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9aa8b-12a8-4df5-a934-ac516bc6cc3d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:26.873574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:26.878349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628782a-25e6-4f74-af73-d3a0008d12de None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:26.878834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628782a-25e6-4f74-af73-d3a0008d12de None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:26.879716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628782a-25e6-4f74-af73-d3a0008d12de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:26.899895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-1628782a-25e6-4f74-af73-d3a0008d12de None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:26.901114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1628782a-25e6-4f74-af73-d3a0008d12de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:26.908854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:27.061689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9aa8b-12a8-4df5-a934-ac516bc6cc3d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:27.081464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9aa8b-12a8-4df5-a934-ac516bc6cc3d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:27.088266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95bcfe-d8ce-43de-9a92-c9c04b14fa6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:27.088928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95bcfe-d8ce-43de-9a92-c9c04b14fa6c service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:27.089215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95bcfe-d8ce-43de-9a92-c9c04b14fa6c service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:27.089506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95bcfe-d8ce-43de-9a92-c9c04b14fa6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:27.098180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95bcfe-d8ce-43de-9a92-c9c04b14fa6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:27.099966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95bcfe-d8ce-43de-9a92-c9c04b14fa6c service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:27.110467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 770/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:27.655530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc1e3-f4d3-4b4d-9160-68c39d567335 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:27.655819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc1e3-f4d3-4b4d-9160-68c39d567335 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:27.656100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc1e3-f4d3-4b4d-9160-68c39d567335 None None] PATH_INFO: `/v3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:27.657436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 747/1517] 10.222.0.20 () {54 vars in 925 bytes} [Fri Jul 29 06:24:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:24:27.663639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603a71c-7b96-4258-9ba7-130332fc1ada None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:27.663936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603a71c-7b96-4258-9ba7-130332fc1ada None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:27.664219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603a71c-7b96-4258-9ba7-130332fc1ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:27.726023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2603a71c-7b96-4258-9ba7-130332fc1ada None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:27.727224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2603a71c-7b96-4258-9ba7-130332fc1ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:27.744224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:24:27.798716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 771/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jul 29 06:24:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:31.576170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5187fb-108b-4e03-b428-bb69cb3d3f12 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:31.576734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5187fb-108b-4e03-b428-bb69cb3d3f12 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:31.576734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5187fb-108b-4e03-b428-bb69cb3d3f12 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:31.578013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 748/1519] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jul 29 06:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:31.583037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d8892-64a9-4c48-87f1-941690bb7d6f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:31.583453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d8892-64a9-4c48-87f1-941690bb7d6f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:31.583599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d8892-64a9-4c48-87f1-941690bb7d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:31.603316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-321d8892-64a9-4c48-87f1-941690bb7d6f None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:31.604522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-321d8892-64a9-4c48-87f1-941690bb7d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:31.613109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 772/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jul 29 06:24:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:31.621375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5187fb-108b-4e03-b428-bb69cb3d3f12 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:31.634766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5187fb-108b-4e03-b428-bb69cb3d3f12 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:31.639642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f33d96-e2fd-4a5f-a6cb-a15fc6942d41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:31.640241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f33d96-e2fd-4a5f-a6cb-a15fc6942d41 service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:31.640525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f33d96-e2fd-4a5f-a6cb-a15fc6942d41 service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:31.640831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f33d96-e2fd-4a5f-a6cb-a15fc6942d41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:31.649443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f33d96-e2fd-4a5f-a6cb-a15fc6942d41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:31.651001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f33d96-e2fd-4a5f-a6cb-a15fc6942d41 service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:31.661218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 749/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:38.352384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724840e2-75f0-4190-8bd4-0202e97cdee5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:38.353017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724840e2-75f0-4190-8bd4-0202e97cdee5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:38.353017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724840e2-75f0-4190-8bd4-0202e97cdee5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:38.354348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:38.359240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe57dc-5aa0-44d1-ac75-be33255871f3 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:38.359711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe57dc-5aa0-44d1-ac75-be33255871f3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:38.359804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe57dc-5aa0-44d1-ac75-be33255871f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:38.379866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-cdfe57dc-5aa0-44d1-ac75-be33255871f3 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:38.381086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-cdfe57dc-5aa0-44d1-ac75-be33255871f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:38.388946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:38.543789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724840e2-75f0-4190-8bd4-0202e97cdee5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:38.559261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724840e2-75f0-4190-8bd4-0202e97cdee5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:38.564814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ee53b-ff8d-4bc0-b601-87ccb3ee3068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:38.565426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ee53b-ff8d-4bc0-b601-87ccb3ee3068 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:38.565708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ee53b-ff8d-4bc0-b601-87ccb3ee3068 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:38.566033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ee53b-ff8d-4bc0-b601-87ccb3ee3068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:38.574217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ee53b-ff8d-4bc0-b601-87ccb3ee3068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:38.575792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ee53b-ff8d-4bc0-b601-87ccb3ee3068 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:38.585628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 774/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:39.404092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26b5d9-f0d7-409b-85ad-2dc553be37f2 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:39.404697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26b5d9-f0d7-409b-85ad-2dc553be37f2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:39.404697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26b5d9-f0d7-409b-85ad-2dc553be37f2 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:39.406043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 06:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:39.410748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2b83c-634e-4e8b-8aa5-03d369434b0b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:39.411206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2b83c-634e-4e8b-8aa5-03d369434b0b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:39.411317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2b83c-634e-4e8b-8aa5-03d369434b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:39.430786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6ce2b83c-634e-4e8b-8aa5-03d369434b0b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:39.431982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6ce2b83c-634e-4e8b-8aa5-03d369434b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:39.439897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 06:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:39.589423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26b5d9-f0d7-409b-85ad-2dc553be37f2 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:39.603619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26b5d9-f0d7-409b-85ad-2dc553be37f2 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:39.609081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cdcec-515c-4a23-b2ef-19873270f029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:39.609688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cdcec-515c-4a23-b2ef-19873270f029 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:39.609972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cdcec-515c-4a23-b2ef-19873270f029 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:39.610251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cdcec-515c-4a23-b2ef-19873270f029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:39.618978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cdcec-515c-4a23-b2ef-19873270f029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:39.620560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cdcec-515c-4a23-b2ef-19873270f029 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:39.630337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 752/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:43.365585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688db6e-b155-4263-938d-a884fd259992 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:43.366163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688db6e-b155-4263-938d-a884fd259992 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:43.366281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688db6e-b155-4263-938d-a884fd259992 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:43.367586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:43.372897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364b5ca-4941-42e9-ac11-46d7ef530c40 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:43.373302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364b5ca-4941-42e9-ac11-46d7ef530c40 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:43.373449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364b5ca-4941-42e9-ac11-46d7ef530c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:43.392812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c364b5ca-4941-42e9-ac11-46d7ef530c40 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:43.395031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c364b5ca-4941-42e9-ac11-46d7ef530c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:43.402908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:43.475682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688db6e-b155-4263-938d-a884fd259992 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:43.477456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5688db6e-b155-4263-938d-a884fd259992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:43.554814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688db6e-b155-4263-938d-a884fd259992 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:43.558351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312310c0-1b6d-49f1-b66e-079c80058286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:43.558967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312310c0-1b6d-49f1-b66e-079c80058286 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:43.559314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312310c0-1b6d-49f1-b66e-079c80058286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:43.559591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312310c0-1b6d-49f1-b66e-079c80058286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:43.569068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312310c0-1b6d-49f1-b66e-079c80058286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:43.570489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312310c0-1b6d-49f1-b66e-079c80058286 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:43.579665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 777/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:45.575117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c72a-95a9-4341-9162-06e6bf2c4a56 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:45.575698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c72a-95a9-4341-9162-06e6bf2c4a56 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:45.575698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c72a-95a9-4341-9162-06e6bf2c4a56 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:45.577068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:45.582445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e1599-9181-4331-a3e7-2f8e33bfe82c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:45.583090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e1599-9181-4331-a3e7-2f8e33bfe82c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:45.583263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e1599-9181-4331-a3e7-2f8e33bfe82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:45.602747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-581e1599-9181-4331-a3e7-2f8e33bfe82c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:45.603948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-581e1599-9181-4331-a3e7-2f8e33bfe82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:45.611945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:45.647954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c72a-95a9-4341-9162-06e6bf2c4a56 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:45.658589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c72a-95a9-4341-9162-06e6bf2c4a56 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:45.662194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72d466-339d-449a-8f92-a178d556a466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:45.662798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72d466-339d-449a-8f92-a178d556a466 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:45.663080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72d466-339d-449a-8f92-a178d556a466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:45.663354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72d466-339d-449a-8f92-a178d556a466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:45.672551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72d466-339d-449a-8f92-a178d556a466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:45.674005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72d466-339d-449a-8f92-a178d556a466 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:45.684045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 755/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:49.213517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73503223-e0a2-4dfb-9aae-90f752058d8f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:49.214083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73503223-e0a2-4dfb-9aae-90f752058d8f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:49.214083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73503223-e0a2-4dfb-9aae-90f752058d8f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:49.215441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:49.220699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e5ee7-b017-44c6-a7dd-5a3c4e70b587 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:49.221128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e5ee7-b017-44c6-a7dd-5a3c4e70b587 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:49.221266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e5ee7-b017-44c6-a7dd-5a3c4e70b587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:49.229544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=81 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:24:49.242449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ab9e5ee7-b017-44c6-a7dd-5a3c4e70b587 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:49.243725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ab9e5ee7-b017-44c6-a7dd-5a3c4e70b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:49.251643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:49.559447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73503223-e0a2-4dfb-9aae-90f752058d8f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:49.561175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-73503223-e0a2-4dfb-9aae-90f752058d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:49.626722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:24:49.640878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73503223-e0a2-4dfb-9aae-90f752058d8f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:49.645439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cd1b6-e491-44e7-aaf1-0650cfcad1f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:49.646054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd1b6-e491-44e7-aaf1-0650cfcad1f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:49.646339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd1b6-e491-44e7-aaf1-0650cfcad1f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:49.646617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd1b6-e491-44e7-aaf1-0650cfcad1f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:49.655541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cd1b6-e491-44e7-aaf1-0650cfcad1f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:49.657032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cd1b6-e491-44e7-aaf1-0650cfcad1f9 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:49.666655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 780/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:24:51.757496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0d562-9669-495a-8e3e-a7e83e9b8b00 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:51.758020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0d562-9669-495a-8e3e-a7e83e9b8b00 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:51.758146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0d562-9669-495a-8e3e-a7e83e9b8b00 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:51.759440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 06:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:24:51.764553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bcf1c-3cdd-4204-90b7-2047b0cbf742 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:51.764952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bcf1c-3cdd-4204-90b7-2047b0cbf742 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:51.765118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bcf1c-3cdd-4204-90b7-2047b0cbf742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:51.786108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3b5bcf1c-3cdd-4204-90b7-2047b0cbf742 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:24:51.787303 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3b5bcf1c-3cdd-4204-90b7-2047b0cbf742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:51.795114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 06:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:24:52.112720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0d562-9669-495a-8e3e-a7e83e9b8b00 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:24:52.115041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d2c0d562-9669-495a-8e3e-a7e83e9b8b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:24:52.128752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0d562-9669-495a-8e3e-a7e83e9b8b00 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:24:52.132393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfc49e-603d-4adf-8640-32320e87339e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:24:52.133022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc49e-603d-4adf-8640-32320e87339e service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:24:52.133302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc49e-603d-4adf-8640-32320e87339e service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:24:52.133582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc49e-603d-4adf-8640-32320e87339e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:24:52.142317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfc49e-603d-4adf-8640-32320e87339e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:24:52.144566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfc49e-603d-4adf-8640-32320e87339e service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:24:52.153874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 758/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:24:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:25:01.780458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=11 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:25:02.149846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=8 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:20.723025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707d60-2f9a-4691-ae29-475ce96616e2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:20.724251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707d60-2f9a-4691-ae29-475ce96616e2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:20.724251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707d60-2f9a-4691-ae29-475ce96616e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:20.758033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cd707d60-2f9a-4691-ae29-475ce96616e2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:20.759301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cd707d60-2f9a-4691-ae29-475ce96616e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:20.771344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 782/1540] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:26:20.780652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e71d7-b5b0-4c1d-8001-1617e05f2a14 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:20.781260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e71d7-b5b0-4c1d-8001-1617e05f2a14 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:20.781459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e71d7-b5b0-4c1d-8001-1617e05f2a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:20.809588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-878e71d7-b5b0-4c1d-8001-1617e05f2a14 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:20.810815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-878e71d7-b5b0-4c1d-8001-1617e05f2a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:20.826701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 759/1541] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:20.831954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707d60-2f9a-4691-ae29-475ce96616e2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:20.843477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707d60-2f9a-4691-ae29-475ce96616e2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:20.853133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:20.854123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:20.854577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:20.855011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:20.855900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:20.856382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:20.858280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:20.877295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 783/1542] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:26:20.884130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b805096-533a-4b78-ad5b-044317233d22 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:20.884426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b805096-533a-4b78-ad5b-044317233d22 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:20.884738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b805096-533a-4b78-ad5b-044317233d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:20.886503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:20.898173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c122324-7e9b-4a6f-87f1-4d0f78f923b3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:20.909792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:20.910734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:20.911196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:20.911656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:20.912518 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:20.914020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:26:20.914020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:26:20.914020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:26:20.915659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:20.919988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-4b805096-533a-4b78-ad5b-044317233d22 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:20.921605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4b805096-533a-4b78-ad5b-044317233d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:20.940690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 760/1543] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:20.949757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0bb5-656c-4abd-8ce3-65d791bb85e2 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:20.950396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0bb5-656c-4abd-8ce3-65d791bb85e2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:20.950934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0bb5-656c-4abd-8ce3-65d791bb85e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:20.981210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d7fd0bb5-656c-4abd-8ce3-65d791bb85e2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:20.982796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d7fd0bb5-656c-4abd-8ce3-65d791bb85e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:20.984972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 784/1544] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:20 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:26:20.986602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.079061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb5da-d799-49f4-bac7-fc555a09d31a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.088385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.091510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.092024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.092517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.093026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 761/1545] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:26:21.095674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.097104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c75d-f44f-473c-9014-653010cfcc8a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.098407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.098881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c75d-f44f-473c-9014-653010cfcc8a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.099397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c75d-f44f-473c-9014-653010cfcc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.137158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9811c75d-f44f-473c-9014-653010cfcc8a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.138505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9811c75d-f44f-473c-9014-653010cfcc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.143645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 785/1546] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:20 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:21.149295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.162359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936eb533-569b-4abb-b115-d87d83fb4d2c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.162891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 762/1547] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.167280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811c75d-f44f-473c-9014-653010cfcc8a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.175087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46959b-5de6-48f4-af69-3269a8227097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.176043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46959b-5de6-48f4-af69-3269a8227097 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.176523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46959b-5de6-48f4-af69-3269a8227097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.177035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46959b-5de6-48f4-af69-3269a8227097 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.178209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46959b-5de6-48f4-af69-3269a8227097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:21.178748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46959b-5de6-48f4-af69-3269a8227097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.180927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46959b-5de6-48f4-af69-3269a8227097 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.181135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811c75d-f44f-473c-9014-653010cfcc8a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.202830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 786/1548] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:21.208739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a3c9-04e5-46d4-97ee-13155ea07be7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.209437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a3c9-04e5-46d4-97ee-13155ea07be7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.209951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a3c9-04e5-46d4-97ee-13155ea07be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.211162 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.211816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.212104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.212388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.213120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.214730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.233533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 763/1549] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:21.234327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.244563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2877a3c9-04e5-46d4-97ee-13155ea07be7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.246696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2877a3c9-04e5-46d4-97ee-13155ea07be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.252467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97bfd6-94a1-424f-9a8d-1776e54349cb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.258055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 787/1550] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.259219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a3c9-04e5-46d4-97ee-13155ea07be7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.274053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.274716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.275005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.275287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.275971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.276252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a3c9-04e5-46d4-97ee-13155ea07be7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.280340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.285422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24f951-4fd5-410e-99a0-e5b7e528d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.286332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24f951-4fd5-410e-99a0-e5b7e528d92d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.286829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24f951-4fd5-410e-99a0-e5b7e528d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.287333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24f951-4fd5-410e-99a0-e5b7e528d92d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.288263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24f951-4fd5-410e-99a0-e5b7e528d92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.290119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24f951-4fd5-410e-99a0-e5b7e528d92d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.302686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 788/1551] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:21.304373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac35d-3cf2-4d59-9ec4-86fa585b9a5d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.304925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac35d-3cf2-4d59-9ec4-86fa585b9a5d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.305437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac35d-3cf2-4d59-9ec4-86fa585b9a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.319478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 764/1552] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/projects => generated 393 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:21.320420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.328572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-273ac35d-3cf2-4d59-9ec4-86fa585b9a5d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.329990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-273ac35d-3cf2-4d59-9ec4-86fa585b9a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.394351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2405e0-03fd-43f1-a6d8-467eb1cb160a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.398509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 789/1553] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.399574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.399831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dff6e7-f0cb-4d77-a92f-364d1f456869 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.400154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dff6e7-f0cb-4d77-a92f-364d1f456869 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.400391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dff6e7-f0cb-4d77-a92f-364d1f456869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.401908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.402362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.402975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.403849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:21.404297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.405680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING py.warnings [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:26:21.405680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: warnings.warn(msg) Jul 29 06:26:21.405680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: Jul 29 06:26:21.410006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b232f86-d561-459f-bfeb-b2e854fda502 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.422694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-81dff6e7-f0cb-4d77-a92f-364d1f456869 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.424823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-81dff6e7-f0cb-4d77-a92f-364d1f456869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.432600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 765/1554] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:21.432600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d24a-34a5-4d96-9ed1-1a4a6f840602 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.432600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d24a-34a5-4d96-9ed1-1a4a6f840602 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.432600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d24a-34a5-4d96-9ed1-1a4a6f840602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.434575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 790/1555] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.436589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020814-0cdf-401c-a40b-63da1e4802a5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.436855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020814-0cdf-401c-a40b-63da1e4802a5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.437133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020814-0cdf-401c-a40b-63da1e4802a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.447572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:21.455659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c154d24a-34a5-4d96-9ed1-1a4a6f840602 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.456912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c154d24a-34a5-4d96-9ed1-1a4a6f840602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.462165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-89020814-0cdf-401c-a40b-63da1e4802a5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.463320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-89020814-0cdf-401c-a40b-63da1e4802a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.465202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 766/1556] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.465951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154d24a-34a5-4d96-9ed1-1a4a6f840602 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.471483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 791/1557] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.472505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020814-0cdf-401c-a40b-63da1e4802a5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.478655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154d24a-34a5-4d96-9ed1-1a4a6f840602 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.481890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020814-0cdf-401c-a40b-63da1e4802a5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.483671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bfbd4-9b5b-4104-b2f4-382aeb0f0320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.485042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bfbd4-9b5b-4104-b2f4-382aeb0f0320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.485351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bfbd4-9b5b-4104-b2f4-382aeb0f0320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.486251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bfbd4-9b5b-4104-b2f4-382aeb0f0320 admin admin] PATH_INFO: `/v3/projects/5ed8bbfcf5ec4f74baf644f4a9d5f200/users/d1e3dde3b1fd48f89f553ccf1dc76a13/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.487546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404001b7-cfb7-494a-83c1-312ac165f942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.488237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404001b7-cfb7-494a-83c1-312ac165f942 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.488518 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404001b7-cfb7-494a-83c1-312ac165f942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.488837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404001b7-cfb7-494a-83c1-312ac165f942 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.489558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404001b7-cfb7-494a-83c1-312ac165f942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.491621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404001b7-cfb7-494a-83c1-312ac165f942 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.506353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bfbd4-9b5b-4104-b2f4-382aeb0f0320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed8bbfcf5ec4f74baf644f4a9d5f200, user_id=d1e3dde3b1fd48f89f553ccf1dc76a13, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.509473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bfbd4-9b5b-4104-b2f4-382aeb0f0320 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.515400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 792/1558] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:21.516903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b4e6-9b4f-40c6-a647-6b1b010fd914 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.517195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b4e6-9b4f-40c6-a647-6b1b010fd914 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.517698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b4e6-9b4f-40c6-a647-6b1b010fd914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.525784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 767/1559] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:21 2022] PUT /identity/v3/projects/5ed8bbfcf5ec4f74baf644f4a9d5f200/users/d1e3dde3b1fd48f89f553ccf1dc76a13/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:21.529455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3ab09-42fa-423d-9c27-31ffa207cbe4 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.529455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3ab09-42fa-423d-9c27-31ffa207cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.529455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3ab09-42fa-423d-9c27-31ffa207cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.541650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8567b4e6-9b4f-40c6-a647-6b1b010fd914 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.542864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8567b4e6-9b4f-40c6-a647-6b1b010fd914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.546995 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-63d3ab09-42fa-423d-9c27-31ffa207cbe4 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.549451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-63d3ab09-42fa-423d-9c27-31ffa207cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.554971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:21.605212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 793/1560] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.606026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b4e6-9b4f-40c6-a647-6b1b010fd914 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.615817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b4e6-9b4f-40c6-a647-6b1b010fd914 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.616293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 768/1561] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.617540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494e25-b2b9-4e93-8baf-4494199d765d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.617840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494e25-b2b9-4e93-8baf-4494199d765d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.618125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2494e25-b2b9-4e93-8baf-4494199d765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.620674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507057da-1336-4c62-a09f-7d8904dd5a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.621284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507057da-1336-4c62-a09f-7d8904dd5a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.621562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507057da-1336-4c62-a09f-7d8904dd5a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.621865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507057da-1336-4c62-a09f-7d8904dd5a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.622543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507057da-1336-4c62-a09f-7d8904dd5a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.624623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507057da-1336-4c62-a09f-7d8904dd5a1e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.637376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d2494e25-b2b9-4e93-8baf-4494199d765d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.638590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d2494e25-b2b9-4e93-8baf-4494199d765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.647563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 769/1562] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.648787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fae66-0575-4a28-a2cc-d9bf084e2581 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.649077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fae66-0575-4a28-a2cc-d9bf084e2581 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.649361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fae66-0575-4a28-a2cc-d9bf084e2581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.655935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 794/1563] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:21.657951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e43182-80ff-4bbe-94f4-bdd47dd99f92 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.658246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e43182-80ff-4bbe-94f4-bdd47dd99f92 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.658523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e43182-80ff-4bbe-94f4-bdd47dd99f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.669711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-270fae66-0575-4a28-a2cc-d9bf084e2581 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.671077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-270fae66-0575-4a28-a2cc-d9bf084e2581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.677263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a2e43182-80ff-4bbe-94f4-bdd47dd99f92 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.678423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a2e43182-80ff-4bbe-94f4-bdd47dd99f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.733473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 770/1564] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.735093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fae66-0575-4a28-a2cc-d9bf084e2581 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.741707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 795/1565] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.742963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b488e0-cdcd-46de-ad4c-aad1d31ad348 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.743258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b488e0-cdcd-46de-ad4c-aad1d31ad348 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.743530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b488e0-cdcd-46de-ad4c-aad1d31ad348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.744713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fae66-0575-4a28-a2cc-d9bf084e2581 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.749413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea8eee-467b-42fe-82eb-1898355fd87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.750026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8eee-467b-42fe-82eb-1898355fd87d admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.750308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8eee-467b-42fe-82eb-1898355fd87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.750581 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8eee-467b-42fe-82eb-1898355fd87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.751808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea8eee-467b-42fe-82eb-1898355fd87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.753430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea8eee-467b-42fe-82eb-1898355fd87d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.762289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a6b488e0-cdcd-46de-ad4c-aad1d31ad348 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.763360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 771/1566] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:21.763464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a6b488e0-cdcd-46de-ad4c-aad1d31ad348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.764642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278302cb-c2ba-48b8-8b36-8061e9fc845d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.764939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278302cb-c2ba-48b8-8b36-8061e9fc845d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.765221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278302cb-c2ba-48b8-8b36-8061e9fc845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.771965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 796/1567] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.772943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b488e0-cdcd-46de-ad4c-aad1d31ad348 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.782696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b488e0-cdcd-46de-ad4c-aad1d31ad348 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.787251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.787407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-278302cb-c2ba-48b8-8b36-8061e9fc845d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:21.787884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.788129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.788400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.788654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-278302cb-c2ba-48b8-8b36-8061e9fc845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:21.789119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:21.789422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.791005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.797289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 772/1568] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:21.798704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278302cb-c2ba-48b8-8b36-8061e9fc845d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.803000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 797/1569] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:21.803693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.809969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278302cb-c2ba-48b8-8b36-8061e9fc845d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.813057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0b1d2-fe22-4134-b00d-c8d231a6d3a4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.815462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.816113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.816397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.816711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.817403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:21.817706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.818402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.819000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.819280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.819401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.819556 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.820203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.822119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.831915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 773/1570] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:21.833311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.843730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e9eda-2e25-4f87-8399-87eb5bc50a09 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.848664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.849277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.850211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.850496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.851256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:21.851567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.852481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 798/1571] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:21.853300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.853436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.864700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280ef36-690b-4443-ad40-ab102595a754 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.872857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 774/1572] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:21.872857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.876599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba424e8-727a-4b22-9149-83a1b0dd1c3a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.877391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.878003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.878283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.878558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.879240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:21.879543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.881198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.881384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.881979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.882265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.882544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.883228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.884826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.893564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 799/1573] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:21.894355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.898720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 775/1574] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:21.898720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.909718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:21.910355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568bb70-beda-4c26-891d-a219989478e4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.915531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.916356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.916671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.917004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.917683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:21.918021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.919668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.932417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 776/1575] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:21.933213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.944485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b297ea2-c1f5-4ad4-830a-b2b884c60c9f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.949311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.950015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.950387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.950754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.951472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.953448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.957986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15095010-a287-4a58-85c8-0b76e57bf112 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:21.961663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:21.962283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:21.962567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:21.962847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:21.972966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:21.974624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:21.980739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 777/1576] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:21.981794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.984198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 800/1577] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:21.985058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:21.996396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041418b-6e42-40f6-b3e7-2c7eca30e218 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.000833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:22.053487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588b909-7cd9-4f4d-b4df-006012c50491 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.058555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.059410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.059783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.060128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.061151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.062847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.069256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.070134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.070426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.070734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.071465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.073835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.089180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 801/1578] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.090264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.098833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 778/1579] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:21 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.098833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.105074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6f44-af67-498e-8ce0-494dec7bd273 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.180196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf2101-a42b-420e-a10f-61968ad86dc0 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.185974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.186754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.187044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.187417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] PATH_INFO: `/v3/projects/872e2c13f9b346598c261de5609fdf89/users/b34e3e2e2e6049428295588959ca595c/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.191076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.192668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.193621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.193912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.194725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.197657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.203657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872e2c13f9b346598c261de5609fdf89, user_id=b34e3e2e2e6049428295588959ca595c, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.206538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.225661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 779/1580] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:22 2022] PUT /identity/v3/projects/872e2c13f9b346598c261de5609fdf89/users/b34e3e2e2e6049428295588959ca595c/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:22.225661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.237980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 802/1581] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.237980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.273529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:22.310842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282ab3b-b088-471a-b2c4-6c36c462bb63 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.316648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.316999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac0f0-7b28-478d-a33d-799a12a7b1d3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.317361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.317663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.317943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.318613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.321116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.324208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.324664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.325057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.325399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.326169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.328253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.354990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 803/1582] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.355945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.370415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 780/1583] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.371394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.387255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:22.455470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf241d-90af-49fb-8893-2fd6b811e76e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.457593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6efc46-d46f-4c42-9093-d7d87d851c34 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.460653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ba02f-b1bc-468f-93d4-3f6592c6268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.461402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ba02f-b1bc-468f-93d4-3f6592c6268e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.461688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ba02f-b1bc-468f-93d4-3f6592c6268e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.461962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ba02f-b1bc-468f-93d4-3f6592c6268e admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.462620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ba02f-b1bc-468f-93d4-3f6592c6268e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.463737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.464169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.464532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.464670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ba02f-b1bc-468f-93d4-3f6592c6268e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.464910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.465691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.467795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.482197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 781/1584] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.483043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.492023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 804/1585] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/users => generated 281 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.493338 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b53dcc-59b2-4dac-8fca-52d79e96fe23 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.494184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f214f8-54ce-4bbd-89a6-b6ce1543cdec None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.494651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f214f8-54ce-4bbd-89a6-b6ce1543cdec None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.495490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f214f8-54ce-4bbd-89a6-b6ce1543cdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.499090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.499724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.499993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.500263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.501644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.502761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING py.warnings [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:26:22.502761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: warnings.warn(msg) Jul 29 06:26:22.502761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: Jul 29 06:26:22.504142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.516621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e9f214f8-54ce-4bbd-89a6-b6ce1543cdec None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:22.518690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9f214f8-54ce-4bbd-89a6-b6ce1543cdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:22.527766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 782/1586] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.528464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.540711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 805/1587] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:22.541700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f214f8-54ce-4bbd-89a6-b6ce1543cdec None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.543727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7911281-4a6a-43f0-86e8-b095db9a963d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.551116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.551781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.552072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.552523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f214f8-54ce-4bbd-89a6-b6ce1543cdec None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.552664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.553324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.555736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.558145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.558574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.558904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.559341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.560071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.562072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.582330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 783/1588] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/users => generated 291 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.583267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.591381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 806/1589] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.591381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.597308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50bbc0-4a47-4a77-90d8-a6802d77fece admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.600378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64a51-3be1-4181-aae8-1ae7a9cbf256 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.602345 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.603092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.603480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.603887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.604633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.604803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.605427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.605718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.605997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] PATH_INFO: `/v3/projects/3b9457137fc44f718e0937bf290a6c2c/users/bb8fc2726c424abca15ad3197f1e7f38/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.607189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.623402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9457137fc44f718e0937bf290a6c2c, user_id=bb8fc2726c424abca15ad3197f1e7f38, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.625043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:26:22.625043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:26:22.625043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:26:22.628059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.635972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 784/1590] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/users => generated 294 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:22.635972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.646184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2dbc0-4c1d-4f14-bf6b-eae8701acf8a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.658778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 807/1591] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:22 2022] PUT /identity/v3/projects/3b9457137fc44f718e0937bf290a6c2c/users/bb8fc2726c424abca15ad3197f1e7f38/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:22.658778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.731968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.733120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.733120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.733303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.733878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:22.734007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.736529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.738956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae2666-2980-4179-b98b-bfc36e2ed9b2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.748418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 785/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.752173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.752173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.752173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.752173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.753659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.755050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.764705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.769725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 808/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.769725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.779376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150956b-5e32-4192-9d8a-c98b5b91e348 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.786251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.786962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.787318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.787666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.788429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.792310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.811550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 809/1594] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.812918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.828331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df718f6-97d0-4ee3-920b-744b35b11e22 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.832441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:22.835198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.835927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.836462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.837013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.837665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3e272-e931-4021-ac3d-1437ec998582 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.837931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.840376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.845473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6435-f983-45db-8930-2ec48fe93b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.846704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6435-f983-45db-8930-2ec48fe93b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.847275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6435-f983-45db-8930-2ec48fe93b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.847749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6435-f983-45db-8930-2ec48fe93b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.862212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 810/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.863376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.864410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6435-f983-45db-8930-2ec48fe93b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.866133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6435-f983-45db-8930-2ec48fe93b38 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.875402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d109e-7ec4-480f-9fd9-33619a24c23c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.878600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 786/1596] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:22.879987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469316-7b9e-4181-97e0-2d574067f653 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.880350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469316-7b9e-4181-97e0-2d574067f653 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.880467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.880793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469316-7b9e-4181-97e0-2d574067f653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.881120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.881406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.881683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.882423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.884120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.903848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 811/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.904927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.918886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0990b20-fc96-4bb4-b77b-3f714c76b894 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.922896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-00469316-7b9e-4181-97e0-2d574067f653 None None] MFA Rules not processed for user `bb8fc2726c424abca15ad3197f1e7f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:22.924476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-00469316-7b9e-4181-97e0-2d574067f653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:22.925026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.925684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.925965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.926241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.927102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.929442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.944636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 812/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.945496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:22.960796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2af06-b9c4-4a40-8574-95c233633498 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:22.967060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:22.967835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:22.968121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:22.968567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:22.969823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:22.973472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:22.992181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 813/1599] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:22.993091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.004914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8094-273a-44f1-a01d-6e5926e10144 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.009773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.010392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.010678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.010955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.011639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.013280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.024567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 814/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:23.025482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.036137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 787/1601] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:23.037046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469316-7b9e-4181-97e0-2d574067f653 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.043506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82c3b-12d6-43f7-9cca-8a96b322ca50 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.055149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.055149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.055149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.055149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.055633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.057157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.057853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469316-7b9e-4181-97e0-2d574067f653 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.065671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.066310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.066671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.066957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.067658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.069307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.086874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 815/1602] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:23.089151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.099474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 788/1603] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:23.100516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.112287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e3022-5757-48a4-81ad-5e265517e801 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.116936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363a34-cb2d-43e8-ba1e-b844f74b25f6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.119539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.120229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.120655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.120972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] PATH_INFO: `/v3/projects/215b94d3dc6c43389f03660530d29469/users/6b36ce7fd16445e19f2ce4d93a67c067/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.123344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d7243-bf3c-4cdc-a496-1070a1d2eb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.124597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d7243-bf3c-4cdc-a496-1070a1d2eb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.125039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d7243-bf3c-4cdc-a496-1070a1d2eb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.127500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d7243-bf3c-4cdc-a496-1070a1d2eb83 admin admin] PATH_INFO: `/v3/projects/a3b773b1bba540f79c21b042d5cf91de/users/e1b9767f26254a6cba945f4cae562b70/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.146359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=215b94d3dc6c43389f03660530d29469, user_id=6b36ce7fd16445e19f2ce4d93a67c067, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.148027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d7243-bf3c-4cdc-a496-1070a1d2eb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b773b1bba540f79c21b042d5cf91de, user_id=e1b9767f26254a6cba945f4cae562b70, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.149328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.151165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d7243-bf3c-4cdc-a496-1070a1d2eb83 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.168972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 816/1604] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:23 2022] PUT /identity/v3/projects/215b94d3dc6c43389f03660530d29469/users/6b36ce7fd16445e19f2ce4d93a67c067/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:23.170393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.174689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 789/1605] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:23 2022] PUT /identity/v3/projects/a3b773b1bba540f79c21b042d5cf91de/users/e1b9767f26254a6cba945f4cae562b70/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:23.176528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c539b9d-7d64-4358-b13a-07f0a9f82596 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.176845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c539b9d-7d64-4358-b13a-07f0a9f82596 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.177124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c539b9d-7d64-4358-b13a-07f0a9f82596 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.179088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 790/1606] 10.222.0.33 () {54 vars in 962 bytes} [Fri Jul 29 06:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:23.179498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c539b9d-7d64-4358-b13a-07f0a9f82596 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.270633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f5798-d40e-460f-bc8a-042040b15023 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.302523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c539b9d-7d64-4358-b13a-07f0a9f82596 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.310577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.311425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.312142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.312569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] PATH_INFO: `/v3/projects/bd0ca31db2cb4b4da308a70b6954629c/users/6f4c287c2adf4196b3d0211cef42afe5/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.336217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0ca31db2cb4b4da308a70b6954629c, user_id=6f4c287c2adf4196b3d0211cef42afe5, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.340380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.359341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab22ab-1a03-48ef-b6eb-69faa578e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.360265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab22ab-1a03-48ef-b6eb-69faa578e93a admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.360764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab22ab-1a03-48ef-b6eb-69faa578e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.361174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab22ab-1a03-48ef-b6eb-69faa578e93a admin admin] PATH_INFO: `/v3/projects/86746df3894446d6a89a199b7962187c/users/cad7dcbf9662422f9b93a2eea25c3584/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.364076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 791/1607] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:23 2022] PUT /identity/v3/projects/bd0ca31db2cb4b4da308a70b6954629c/users/6f4c287c2adf4196b3d0211cef42afe5/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:23.364607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:23.365739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.392397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab22ab-1a03-48ef-b6eb-69faa578e93a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86746df3894446d6a89a199b7962187c, user_id=cad7dcbf9662422f9b93a2eea25c3584, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.395287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab22ab-1a03-48ef-b6eb-69faa578e93a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.423778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 817/1608] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:23 2022] PUT /identity/v3/projects/86746df3894446d6a89a199b7962187c/users/cad7dcbf9662422f9b93a2eea25c3584/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:23.437655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07db2-ba7c-4a4b-b0c9-3c4a92dd6183 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.437949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07db2-ba7c-4a4b-b0c9-3c4a92dd6183 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.439674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:23.440163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07db2-ba7c-4a4b-b0c9-3c4a92dd6183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.469263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d9f07db2-ba7c-4a4b-b0c9-3c4a92dd6183 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:23.471856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d9f07db2-ba7c-4a4b-b0c9-3c4a92dd6183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:23.494403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55be3d-4632-4814-a0aa-24369b64383f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:23.571760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 818/1609] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:23.584254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be225e8b-4d58-4dc8-9087-9949401f85f1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.584662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be225e8b-4d58-4dc8-9087-9949401f85f1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.584946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be225e8b-4d58-4dc8-9087-9949401f85f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.605666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1046b7-80d0-4517-8b38-d004efa7b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:23.606450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1046b7-80d0-4517-8b38-d004efa7b04f admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.606781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1046b7-80d0-4517-8b38-d004efa7b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.607061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1046b7-80d0-4517-8b38-d004efa7b04f admin admin] PATH_INFO: `/v3/projects/f8d1f62a52884ebb91749ef49de11bec/users/217b321936224e2ba928b983c4e40017/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.628458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-be225e8b-4d58-4dc8-9087-9949401f85f1 None None] MFA Rules not processed for user `6b36ce7fd16445e19f2ce4d93a67c067`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:23.629621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1046b7-80d0-4517-8b38-d004efa7b04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d1f62a52884ebb91749ef49de11bec, user_id=217b321936224e2ba928b983c4e40017, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:23.630985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-be225e8b-4d58-4dc8-9087-9949401f85f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:23.634793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1046b7-80d0-4517-8b38-d004efa7b04f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:23.662935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 792/1610] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:23 2022] PUT /identity/v3/projects/f8d1f62a52884ebb91749ef49de11bec/users/217b321936224e2ba928b983c4e40017/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:23.664815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af6a0d-9032-4294-b861-bfcfca6144ec None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.665133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af6a0d-9032-4294-b861-bfcfca6144ec None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.665989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af6a0d-9032-4294-b861-bfcfca6144ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.711603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-74af6a0d-9032-4294-b861-bfcfca6144ec None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:23.712452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 819/1611] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:23.712889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-74af6a0d-9032-4294-b861-bfcfca6144ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:23.713838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.714149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.714525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.761181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] MFA Rules not processed for user `6f4c287c2adf4196b3d0211cef42afe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:23.763232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:23.816818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 793/1612] 10.222.0.33 () {58 vars in 1059 bytes} [Fri Jul 29 06:26:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:23.818361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401002b-3fc7-42a6-89ae-f31d93825923 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:23.818683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401002b-3fc7-42a6-89ae-f31d93825923 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:23.818957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401002b-3fc7-42a6-89ae-f31d93825923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:23.858861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3401002b-3fc7-42a6-89ae-f31d93825923 None None] MFA Rules not processed for user `cad7dcbf9662422f9b93a2eea25c3584`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:23.860095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3401002b-3fc7-42a6-89ae-f31d93825923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:23.905693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:23.980725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 820/1613] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:23.980725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:23.980725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:23.980725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:24.021866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 794/1614] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:24.023574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65190-9214-4e9b-9ff3-8b4abeda18ed None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.024139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65190-9214-4e9b-9ff3-8b4abeda18ed None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.024621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65190-9214-4e9b-9ff3-8b4abeda18ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.061586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-fde65190-9214-4e9b-9ff3-8b4abeda18ed None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:24.063308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3152a-7adc-4c32-84e7-f429db5a1151 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.067459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.068112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.068353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fde65190-9214-4e9b-9ff3-8b4abeda18ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:24.068482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.068718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.162039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.164294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.179299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 821/1615] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:24.180360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.188160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 795/1616] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:24.189989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde65190-9214-4e9b-9ff3-8b4abeda18ed None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.192131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a2719-d513-4823-aa3d-d0805c57c220 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.199112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.200159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.200645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.200980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.201875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.205029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.210648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde65190-9214-4e9b-9ff3-8b4abeda18ed None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.218158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a66dd-ffdd-490e-8211-5b8e26e3bf5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.219027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a66dd-ffdd-490e-8211-5b8e26e3bf5b service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.219457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a66dd-ffdd-490e-8211-5b8e26e3bf5b service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.220480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a66dd-ffdd-490e-8211-5b8e26e3bf5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.248097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 822/1617] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:24 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:24.249156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.384846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a66dd-ffdd-490e-8211-5b8e26e3bf5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.388166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a66dd-ffdd-490e-8211-5b8e26e3bf5b service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.437891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 796/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:24.441428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b9c7-6875-4f69-a84a-2d62ed18780e None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.443031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b9c7-6875-4f69-a84a-2d62ed18780e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.444010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b9c7-6875-4f69-a84a-2d62ed18780e None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.446288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff14926-d72e-4a7a-9c13-ca556fad4772 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.448505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 797/1619] 10.222.0.33 () {54 vars in 962 bytes} [Fri Jul 29 06:26:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:24.451322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9b9c7-6875-4f69-a84a-2d62ed18780e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.460187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.461036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.461520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.461901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.462747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.467248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.552792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 823/1620] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:24.554924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.589828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:24.603334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9b9c7-6875-4f69-a84a-2d62ed18780e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.649046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:24.666451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c328e67-3145-4302-b2de-b3d606e71f41 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.680340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fc9eb-75aa-4206-970b-03d4b09a9cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.681148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc9eb-75aa-4206-970b-03d4b09a9cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.681529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc9eb-75aa-4206-970b-03d4b09a9cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.681912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc9eb-75aa-4206-970b-03d4b09a9cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.682661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fc9eb-75aa-4206-970b-03d4b09a9cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.687702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fc9eb-75aa-4206-970b-03d4b09a9cf9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.717443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.718586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.719072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.719557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.734222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 824/1621] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:24 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:24.735723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28a9b9-5fa1-4350-a119-818b459639e5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.736963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28a9b9-5fa1-4350-a119-818b459639e5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.737350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28a9b9-5fa1-4350-a119-818b459639e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.741105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.756707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.776326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 798/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:24.777249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.802789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e757397-557d-4b45-aaa9-5cbbd957978b service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.806209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7d28a9b9-5fa1-4350-a119-818b459639e5 None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:24.808187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7d28a9b9-5fa1-4350-a119-818b459639e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:24.818310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.818923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.819210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.820088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.827842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.832116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.889398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 799/1623] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:24 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:24.890393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.908406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2bb2-f3b3-4459-8860-729d5e0b6fd7 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.915477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.916117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.916517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.916965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.918011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.921156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.947155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 800/1624] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:24.948634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.962718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438768e3-b713-4249-86d2-f988831a5606 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:24.968772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:24.969571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.970963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.971465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.972336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:24.974494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:24.978069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 825/1625] 10.222.0.33 () {58 vars in 1059 bytes} [Fri Jul 29 06:26:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:24.979579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc930f-c7d9-45b0-a6c2-da2ea2356725 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:24.979938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc930f-c7d9-45b0-a6c2-da2ea2356725 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:24.980287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc930f-c7d9-45b0-a6c2-da2ea2356725 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:24.981728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 826/1626] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 29 06:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:24.982475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc930f-c7d9-45b0-a6c2-da2ea2356725 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:24.998971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 801/1627] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:25.000002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:25.018817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9282454-3d15-4d1e-89a4-3b25c887905f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.020989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc930f-c7d9-45b0-a6c2-da2ea2356725 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.027418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a0034-4849-437c-b7df-66fd376cbb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.027579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.028420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.028523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a0034-4849-437c-b7df-66fd376cbb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.028990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.029279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.029434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a0034-4849-437c-b7df-66fd376cbb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.030682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a0034-4849-437c-b7df-66fd376cbb81 admin admin] PATH_INFO: `/v3/projects/027ba2fe28384d3789886b03b4e66c06/users/62ededefab4341aca3b9671e01a3f3a9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.066883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a0034-4849-437c-b7df-66fd376cbb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=027ba2fe28384d3789886b03b4e66c06, user_id=62ededefab4341aca3b9671e01a3f3a9, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.075394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a0034-4849-437c-b7df-66fd376cbb81 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.112083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 827/1628] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:24 2022] PUT /identity/v3/projects/027ba2fe28384d3789886b03b4e66c06/users/62ededefab4341aca3b9671e01a3f3a9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:25.114698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fac467-5b4b-495b-807d-43134710c365 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.115052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fac467-5b4b-495b-807d-43134710c365 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.116063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fac467-5b4b-495b-807d-43134710c365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.166280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a6fac467-5b4b-495b-807d-43134710c365 None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:25.169942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a6fac467-5b4b-495b-807d-43134710c365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:25.193578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.199504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.234403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 802/1629] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:25.238426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:25.264860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e508-bfce-4461-9f0c-5918383a7dcf service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.358637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 828/1630] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:26:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:25.359564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fac467-5b4b-495b-807d-43134710c365 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:25.373475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:25.404196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.405670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.408070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.408377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] PATH_INFO: `/v3/projects/fa36c24d35ff4224bbaee304e355de75/users/4b90be14a70a49df856f4d22e4200f0c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.448005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa36c24d35ff4224bbaee304e355de75, user_id=4b90be14a70a49df856f4d22e4200f0c, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.455868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.482496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 803/1631] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:25 2022] PUT /identity/v3/projects/fa36c24d35ff4224bbaee304e355de75/users/4b90be14a70a49df856f4d22e4200f0c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:25.486777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:25.508063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:25.512363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fac467-5b4b-495b-807d-43134710c365 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.606717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf2d29-4d5b-46f3-8cf5-97f92c699fb6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.615045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a1c1f-8558-4a54-a9ec-ea77a0f20339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.616198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a1c1f-8558-4a54-a9ec-ea77a0f20339 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.617271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a1c1f-8558-4a54-a9ec-ea77a0f20339 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.617553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a1c1f-8558-4a54-a9ec-ea77a0f20339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.648988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.650905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.651026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.651306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.659860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.666411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.700472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 829/1632] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:25.701472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:25.736110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28eb9e-8d8e-4748-9277-f7198edfac89 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.748378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.749818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.750144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.750470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.752192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.757232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.771763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a1c1f-8558-4a54-a9ec-ea77a0f20339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.773874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a1c1f-8558-4a54-a9ec-ea77a0f20339 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.782565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 830/1633] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:25.783483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:25.790980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 804/1634] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:25.792303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d90b-86c6-4c03-b0f1-30bf7f8f1b12 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.792625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d90b-86c6-4c03-b0f1-30bf7f8f1b12 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.792909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d90b-86c6-4c03-b0f1-30bf7f8f1b12 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.794205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 805/1635] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 29 06:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:25.799474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6d90b-86c6-4c03-b0f1-30bf7f8f1b12 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:25.805344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a18d51-7a0e-4dfa-a213-d746f36f123c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.815478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6d90b-86c6-4c03-b0f1-30bf7f8f1b12 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:25.818241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a94df-b5f4-43ba-9b61-5ac1d5d5c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.819127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a94df-b5f4-43ba-9b61-5ac1d5d5c8ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.819412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a94df-b5f4-43ba-9b61-5ac1d5d5c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.819691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a94df-b5f4-43ba-9b61-5ac1d5d5c8ff admin admin] PATH_INFO: `/v3/projects/027ba2fe28384d3789886b03b4e66c06/users/62ededefab4341aca3b9671e01a3f3a9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.826495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a94df-b5f4-43ba-9b61-5ac1d5d5c8ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=027ba2fe28384d3789886b03b4e66c06, user_id=62ededefab4341aca3b9671e01a3f3a9, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.826902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8c17-2e80-4527-bbfc-cb67db10d3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:25.828076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8c17-2e80-4527-bbfc-cb67db10d3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.828466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8c17-2e80-4527-bbfc-cb67db10d3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.828790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8c17-2e80-4527-bbfc-cb67db10d3f9 admin admin] PATH_INFO: `/v3/projects/fa36c24d35ff4224bbaee304e355de75/users/4b90be14a70a49df856f4d22e4200f0c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.831787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a94df-b5f4-43ba-9b61-5ac1d5d5c8ff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.833588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8c17-2e80-4527-bbfc-cb67db10d3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa36c24d35ff4224bbaee304e355de75, user_id=4b90be14a70a49df856f4d22e4200f0c, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:25.838096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8c17-2e80-4527-bbfc-cb67db10d3f9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:25.852192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 806/1636] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:25 2022] PUT /identity/v3/projects/fa36c24d35ff4224bbaee304e355de75/users/4b90be14a70a49df856f4d22e4200f0c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:25.864037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d5df-e2ce-4d2d-b26d-0173e7684280 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.864385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d5df-e2ce-4d2d-b26d-0173e7684280 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.864701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d5df-e2ce-4d2d-b26d-0173e7684280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.865865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 831/1637] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:25 2022] PUT /identity/v3/projects/027ba2fe28384d3789886b03b4e66c06/users/62ededefab4341aca3b9671e01a3f3a9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:25.877465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4db0-23ec-4e96-ae90-cf0f11ad656d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:25.877790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4db0-23ec-4e96-ae90-cf0f11ad656d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:25.878046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4db0-23ec-4e96-ae90-cf0f11ad656d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:25.905750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d734d5df-e2ce-4d2d-b26d-0173e7684280 None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:25.910693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d734d5df-e2ce-4d2d-b26d-0173e7684280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:25.911750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7cbb4db0-23ec-4e96-ae90-cf0f11ad656d None None] MFA Rules not processed for user `4b90be14a70a49df856f4d22e4200f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:25.913008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7cbb4db0-23ec-4e96-ae90-cf0f11ad656d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:26.039919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 832/1638] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:26.041543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8b374-28d9-4831-b6b0-ef4a1b78e317 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:26.044703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8b374-28d9-4831-b6b0-ef4a1b78e317 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:26.049562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8b374-28d9-4831-b6b0-ef4a1b78e317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:26.087311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 807/1639] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:26:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:26.105714 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734d5df-e2ce-4d2d-b26d-0173e7684280 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:26.123570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:26.143863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4db8b374-28d9-4831-b6b0-ef4a1b78e317 None None] MFA Rules not processed for user `62ededefab4341aca3b9671e01a3f3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:26.145362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4db8b374-28d9-4831-b6b0-ef4a1b78e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:26.201398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:26.254974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734d5df-e2ce-4d2d-b26d-0173e7684280 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:26.272191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:26.272866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:26.273149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:26.273426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:26.302898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:26.320880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:26.375055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 808/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:26.444958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 833/1641] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:26 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:26.488950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:26.505564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1bd95-c3fc-4a18-9e91-208439f8138d service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:26.521932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:26.524412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:26.530493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:26.532393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:26.586717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:26.589937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:26.618348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 809/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:27.129292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8b374-28d9-4831-b6b0-ef4a1b78e317 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:27.145377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8b374-28d9-4831-b6b0-ef4a1b78e317 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:27.159350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:27.160577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:27.165468 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:27.166089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:27.187336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:27.196495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:27.228321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 834/1643] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:27 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:27.682723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:27.693151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a494c-a66e-4800-a7e2-2621bc10b733 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:27.713052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2aab8-9c78-458e-92cf-b6dd2477bd65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:27.713700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2aab8-9c78-458e-92cf-b6dd2477bd65 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:27.713984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2aab8-9c78-458e-92cf-b6dd2477bd65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:27.714839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2aab8-9c78-458e-92cf-b6dd2477bd65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:27.813456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2aab8-9c78-458e-92cf-b6dd2477bd65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:27.813456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2aab8-9c78-458e-92cf-b6dd2477bd65 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:27.817644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:27.827294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725ca2f-3ccf-4bcc-931f-929a2da8952d service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:27.838260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1cea39-eaec-47b5-a117-33054c8cc005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:27.838953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cea39-eaec-47b5-a117-33054c8cc005 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:27.839238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cea39-eaec-47b5-a117-33054c8cc005 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:27.839516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cea39-eaec-47b5-a117-33054c8cc005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:27.850029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 810/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:26:27.929179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1cea39-eaec-47b5-a117-33054c8cc005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:27.930681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1cea39-eaec-47b5-a117-33054c8cc005 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:27.958056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 835/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:26:30.518720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045793d-a4a9-4e30-8c03-6e53141eef12 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:30.519311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045793d-a4a9-4e30-8c03-6e53141eef12 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:30.519311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045793d-a4a9-4e30-8c03-6e53141eef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:30.609208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2045793d-a4a9-4e30-8c03-6e53141eef12 None None] MFA Rules not processed for user `d1e3dde3b1fd48f89f553ccf1dc76a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:30.611181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2045793d-a4a9-4e30-8c03-6e53141eef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:30.830783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 811/1646] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:30.834836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c70c8-accc-4fe6-868a-f763a48d8b53 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:30.835341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c70c8-accc-4fe6-868a-f763a48d8b53 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:30.836115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c70c8-accc-4fe6-868a-f763a48d8b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:30.867354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a83c70c8-accc-4fe6-868a-f763a48d8b53 None None] MFA Rules not processed for user `d1e3dde3b1fd48f89f553ccf1dc76a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:30.868630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a83c70c8-accc-4fe6-868a-f763a48d8b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:30.881249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 836/1647] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:30.892333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045793d-a4a9-4e30-8c03-6e53141eef12 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:30.931293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045793d-a4a9-4e30-8c03-6e53141eef12 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:30.944230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7008a-77c0-4cff-afbd-7a8f1828d8b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:30.945124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7008a-77c0-4cff-afbd-7a8f1828d8b5 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:30.947979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7008a-77c0-4cff-afbd-7a8f1828d8b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:30.948402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7008a-77c0-4cff-afbd-7a8f1828d8b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:30.964751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7008a-77c0-4cff-afbd-7a8f1828d8b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:30.967396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7008a-77c0-4cff-afbd-7a8f1828d8b5 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:30.986857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 812/1648] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:32.165434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebf9f-5e81-4cb0-87de-06a1d3ed8a18 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:32.166434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebf9f-5e81-4cb0-87de-06a1d3ed8a18 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:32.167161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebf9f-5e81-4cb0-87de-06a1d3ed8a18 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:32.170007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 837/1649] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 29 06:26:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:32.175737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b4eb-43b7-44a6-a3cc-1697cbc47783 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:32.176284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b4eb-43b7-44a6-a3cc-1697cbc47783 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:32.176284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b4eb-43b7-44a6-a3cc-1697cbc47783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:32.257407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3553b4eb-43b7-44a6-a3cc-1697cbc47783 None None] MFA Rules not processed for user `edf0a00538cf4e10b3bf266f48ea8cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:32.258975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3553b4eb-43b7-44a6-a3cc-1697cbc47783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:32.285638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:32.375334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 813/1650] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:26:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:32.427423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ebf9f-5e81-4cb0-87de-06a1d3ed8a18 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:32.445795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ebf9f-5e81-4cb0-87de-06a1d3ed8a18 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:32.458822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:32.459929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:32.460368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:32.460735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:32.479099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:32.482157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:32.507978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 838/1651] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:33.088740 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b4eb-43b7-44a6-a3cc-1697cbc47783 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:33.254205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:33.365739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b4eb-43b7-44a6-a3cc-1697cbc47783 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:33.389255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:33.389939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:33.390223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:33.390501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] PATH_INFO: `/v3/users/bb8fc2726c424abca15ad3197f1e7f38` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:33.395304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8fc2726c424abca15ad3197f1e7f38)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:33.397317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:33.497041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e856071-1d9d-4bf3-b620-3cff45f1238f service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:33.520836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:33.525727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:33.526477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:33.528245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] PATH_INFO: `/v3/users/6b36ce7fd16445e19f2ce4d93a67c067` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:33.534357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b36ce7fd16445e19f2ce4d93a67c067)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:33.538853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:26:33.538853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jul 29 06:26:33.538853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: Jul 29 06:26:33.542626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:33.545685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:33.576242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8fc2726c424abca15ad3197f1e7f38'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.577481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8fc2726c424abca15ad3197f1e7f38'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.638580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8fc2726c424abca15ad3197f1e7f38'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.668654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8fc2726c424abca15ad3197f1e7f38'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.672030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4d7c17a-02e3-4a7d-ab83-e04940d9e8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8fc2726c424abca15ad3197f1e7f38'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.712994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 814/1652] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:33 2022] DELETE /identity/v3/users/bb8fc2726c424abca15ad3197f1e7f38 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:33.723227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa23a-a442-454d-a2e3-ccebb69e634d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:33.726229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa23a-a442-454d-a2e3-ccebb69e634d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:33.728363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa23a-a442-454d-a2e3-ccebb69e634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:33.774442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b36ce7fd16445e19f2ce4d93a67c067'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.806656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-869fa23a-a442-454d-a2e3-ccebb69e634d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:33.815204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-869fa23a-a442-454d-a2e3-ccebb69e634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:33.822852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b36ce7fd16445e19f2ce4d93a67c067'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.858698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b36ce7fd16445e19f2ce4d93a67c067'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.900214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b36ce7fd16445e19f2ce4d93a67c067'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.902247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9a2dad1-72c6-46ac-98ab-88e2f8aaa23d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b36ce7fd16445e19f2ce4d93a67c067'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:33.932324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 839/1653] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:33 2022] DELETE /identity/v3/users/6b36ce7fd16445e19f2ce4d93a67c067 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:33.935391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:33.937478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:33.939939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:33.986432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:34.041913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] MFA Rules not processed for user `b34e3e2e2e6049428295588959ca595c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:34.045149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:34.058834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 815/1654] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:34.060411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484313-0361-4fb4-8832-1a73b8733c60 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.061671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484313-0361-4fb4-8832-1a73b8733c60 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.064779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484313-0361-4fb4-8832-1a73b8733c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.149055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-23484313-0361-4fb4-8832-1a73b8733c60 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:34.150321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-23484313-0361-4fb4-8832-1a73b8733c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:34.303112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 840/1655] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:34.305330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:34.306891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:34.399798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 816/1656] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:34.414614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.414911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.415194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.503435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] MFA Rules not processed for user `b34e3e2e2e6049428295588959ca595c`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:34.504924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:34.534391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462e9b7-9fea-4d6f-9e19-734430a58d3d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:34.536381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 817/1657] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:34.537153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:34.538545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f716aae-029d-44f9-a577-a1ff22d6bd5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:34.539189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f716aae-029d-44f9-a577-a1ff22d6bd5f service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.539472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f716aae-029d-44f9-a577-a1ff22d6bd5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.539751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f716aae-029d-44f9-a577-a1ff22d6bd5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.549659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:34.574570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1d3d3-8b74-43d4-a0b4-645c46e3ca52 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:34.589266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f716aae-029d-44f9-a577-a1ff22d6bd5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:34.589475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:34.589475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.589879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.590007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.595684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f716aae-029d-44f9-a577-a1ff22d6bd5f service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:34.611851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:34.613127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:34.632630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 818/1658] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:34.633497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:34.638408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 841/1659] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:34.639602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b1cac-1e18-4655-8071-b1c68738e46f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.639890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b1cac-1e18-4655-8071-b1c68738e46f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.646098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:34.646276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b1cac-1e18-4655-8071-b1c68738e46f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.654267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 842/1660] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jul 29 06:26:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:34.655681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4290ce3-1a2a-4c06-9e92-b3312faa1747 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.655970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4290ce3-1a2a-4c06-9e92-b3312faa1747 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.662975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4290ce3-1a2a-4c06-9e92-b3312faa1747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.703359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a4290ce3-1a2a-4c06-9e92-b3312faa1747 None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:34.704803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a4290ce3-1a2a-4c06-9e92-b3312faa1747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:34.724712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:34.812180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd94b36-4c7a-4941-956d-f9f121f430fb service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:34.830231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:34.830892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.831181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.831459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.849922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:34.851660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:34.873060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 843/1661] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jul 29 06:26:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:34.874406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4290ce3-1a2a-4c06-9e92-b3312faa1747 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:34.894318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 819/1662] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:34.918707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4290ce3-1a2a-4c06-9e92-b3312faa1747 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:34.977865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:34.979904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:34.980520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:34.981547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:34.986024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:35.018667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:35.021574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:35.056953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 844/1663] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:36.702219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:36.704295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:36.766653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53344cf8-f851-4833-8da7-ff1e5e3019d7 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:36.778574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:36.780479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:36.783110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:36.783494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] PATH_INFO: `/v3/projects/3b9457137fc44f718e0937bf290a6c2c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:36.785656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9457137fc44f718e0937bf290a6c2c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:36.788067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: WARNING py.warnings [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 06:26:36.788067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: warnings.warn(msg) Jul 29 06:26:36.788067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: Jul 29 06:26:36.792016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:36.931204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:36.937392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9457137fc44f718e0937bf290a6c2c', 'request_id': 'req-4f8902c8-5be6-484e-8179-4a1961ea14b2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:36.966160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:36.967754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:36.977839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9457137fc44f718e0937bf290a6c2c', 'request_id': 'req-4f8902c8-5be6-484e-8179-4a1961ea14b2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:36.994214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea672-568b-4fbb-af53-030cd4c027f1 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:37.002576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9457137fc44f718e0937bf290a6c2c', 'request_id': 'req-4f8902c8-5be6-484e-8179-4a1961ea14b2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.026931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] The token cache is being invalidate because project 3b9457137fc44f718e0937bf290a6c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:37.027233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.030499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4f8902c8-5be6-484e-8179-4a1961ea14b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.038082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 820/1664] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:36 2022] DELETE /identity/v3/projects/3b9457137fc44f718e0937bf290a6c2c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:37.039678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf490-eca7-47d9-be6a-8a1f7259e6c1 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:37.039974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf490-eca7-47d9-be6a-8a1f7259e6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:37.040329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf490-eca7-47d9-be6a-8a1f7259e6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:37.089436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-262cf490-eca7-47d9-be6a-8a1f7259e6c1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:37.090886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-262cf490-eca7-47d9-be6a-8a1f7259e6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:37.216179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:37.217041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:37.217564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:37.218184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] PATH_INFO: `/v3/projects/215b94d3dc6c43389f03660530d29469` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:37.220499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=215b94d3dc6c43389f03660530d29469)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:37.225018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:37.332421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 821/1665] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:37.340688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138dd8c-cca3-4b33-9bb3-9c96b9edd53d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:37.341363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138dd8c-cca3-4b33-9bb3-9c96b9edd53d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:37.342161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138dd8c-cca3-4b33-9bb3-9c96b9edd53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:37.390545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c138dd8c-cca3-4b33-9bb3-9c96b9edd53d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:37.393818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c138dd8c-cca3-4b33-9bb3-9c96b9edd53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:37.402137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '215b94d3dc6c43389f03660530d29469', 'request_id': 'req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.440145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '215b94d3dc6c43389f03660530d29469', 'request_id': 'req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.464226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '215b94d3dc6c43389f03660530d29469', 'request_id': 'req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.507228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] The token cache is being invalidate because project 215b94d3dc6c43389f03660530d29469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:37.507715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.512122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.516235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 845/1666] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:36 2022] DELETE /identity/v3/projects/215b94d3dc6c43389f03660530d29469 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:37.522090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:37.523784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:37.546555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 822/1667] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:26:37.548177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138dd8c-cca3-4b33-9bb3-9c96b9edd53d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:37.558722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138dd8c-cca3-4b33-9bb3-9c96b9edd53d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:37.572474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:37.572474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:37.572901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:37.573051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:37.574533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:37.577315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:37.579100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:37.612964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2629ecb-c100-4ee2-b18d-bf28b78e5de6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:37.633736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 823/1668] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:37.635846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:37.636664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:37.637057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:37.637383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:37.637498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] PATH_INFO: `/v3/users/62ededefab4341aca3b9671e01a3f3a9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:37.639946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ededefab4341aca3b9671e01a3f3a9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:37.642620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:37.679344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d88f8d-fa69-4823-a334-88c14d59992d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:37.695441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00692e-f57b-4efd-9e20-01dabebafbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:37.696096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00692e-f57b-4efd-9e20-01dabebafbfd admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:37.696393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00692e-f57b-4efd-9e20-01dabebafbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:37.696707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00692e-f57b-4efd-9e20-01dabebafbfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:37.697391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00692e-f57b-4efd-9e20-01dabebafbfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:37.715062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00692e-f57b-4efd-9e20-01dabebafbfd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:37.757738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:37.849056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ededefab4341aca3b9671e01a3f3a9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.868138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ededefab4341aca3b9671e01a3f3a9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.870116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 824/1669] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:37 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:37.876005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:37.876561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:37.876875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:37.929249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ededefab4341aca3b9671e01a3f3a9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.934742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] MFA Rules not processed for user `b34e3e2e2e6049428295588959ca595c`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:37.937247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:37.976707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ededefab4341aca3b9671e01a3f3a9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:37.977755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ededefab4341aca3b9671e01a3f3a9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:38.000502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 846/1670] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:37 2022] DELETE /identity/v3/users/62ededefab4341aca3b9671e01a3f3a9 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:26:38.002832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:38.006423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:38.098312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:38.190863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 825/1671] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:38.195013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:38.199112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:38.302590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f769-60ce-4de0-80ca-e46b40784fbd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:38.336785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:38.337453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:38.337763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:38.338036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:38.338722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:38.353216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:38.425962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 847/1672] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:38 2022] POST /identity/v3/users => generated 290 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:38.429202 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:38.450966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2c39-8180-4b84-99fe-dde1340e78ff admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:38.474359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:38.475275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:38.478546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:38.486100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:38.488695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:38.493827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16b201-072b-4791-9398-db9226d0fbfb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:38.504401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:38.508346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:38.510285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:38.510680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:38.510971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] PATH_INFO: `/v3/users/6f4c287c2adf4196b3d0211cef42afe5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:38.530729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4c287c2adf4196b3d0211cef42afe5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:38.533313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:38.557444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 848/1673] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:38.561321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:38.599651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c7ba5-aa09-4106-9d07-7a512fada3f9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:38.626945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:38.627604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:38.635265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:38.635265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:38.635265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:38.647302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:38.708550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 849/1674] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:38.711875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4c287c2adf4196b3d0211cef42afe5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:38.712490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:38.741464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4c287c2adf4196b3d0211cef42afe5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:38.741894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ea12-216c-40f8-acc7-6c19edef099d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:38.767545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945fcaf-1ae3-4172-bfe4-18845bd87a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:38.768252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945fcaf-1ae3-4172-bfe4-18845bd87a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:38.768552 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945fcaf-1ae3-4172-bfe4-18845bd87a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:38.773920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945fcaf-1ae3-4172-bfe4-18845bd87a61 admin admin] PATH_INFO: `/v3/projects/51197c74305348a48f4b8d4e5f3ce581/users/76463842343a4332be03b61a48d7e9e3/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:38.788676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4c287c2adf4196b3d0211cef42afe5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:38.800452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4c287c2adf4196b3d0211cef42afe5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:38.807592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6aa4d116-9d11-4b42-9b08-74333c361550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4c287c2adf4196b3d0211cef42afe5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:38.852446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945fcaf-1ae3-4172-bfe4-18845bd87a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51197c74305348a48f4b8d4e5f3ce581, user_id=76463842343a4332be03b61a48d7e9e3, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:38.867393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945fcaf-1ae3-4172-bfe4-18845bd87a61 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:38.867848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 826/1675] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:38 2022] DELETE /identity/v3/users/6f4c287c2adf4196b3d0211cef42afe5 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:38.878866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3bf5-82c7-4350-ad24-f72ac922b6f0 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:38.879163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3bf5-82c7-4350-ad24-f72ac922b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:38.891107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3bf5-82c7-4350-ad24-f72ac922b6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:38.919821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 850/1676] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:38 2022] PUT /identity/v3/projects/51197c74305348a48f4b8d4e5f3ce581/users/76463842343a4332be03b61a48d7e9e3/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:38.934528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:38.934928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:38.935222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:38.949049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-858d3bf5-82c7-4350-ad24-f72ac922b6f0 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:38.950308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-858d3bf5-82c7-4350-ad24-f72ac922b6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:39.049724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] MFA Rules not processed for user `76463842343a4332be03b61a48d7e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:39.051003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:39.107142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 827/1677] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:39.112373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.112941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.113461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:39.144689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:39.162552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:39.211643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] MFA Rules not processed for user `e1b9767f26254a6cba945f4cae562b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:39.213114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:39.268218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 851/1678] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:39.269482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:39.272336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:39.351812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 828/1679] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:39.352995 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:39.355994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:39.394961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e1e05-e772-4387-9d7d-49d4d480facc None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:39.407046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:39.407694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.407984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.408270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:39.409478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:39.410931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:39.456140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea8e-3f0d-424a-97f7-6f8e7732c1a0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:39.460963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33517461-a7c4-4473-8ac4-7e70bb29b0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:39.462338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33517461-a7c4-4473-8ac4-7e70bb29b0bb service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.462855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33517461-a7c4-4473-8ac4-7e70bb29b0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.464133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33517461-a7c4-4473-8ac4-7e70bb29b0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:39.477291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33517461-a7c4-4473-8ac4-7e70bb29b0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:39.479875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33517461-a7c4-4473-8ac4-7e70bb29b0bb service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:39.494993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 852/1680] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:39.497393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b5504-92a0-4b4f-88cd-6ac0318e85b2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.497934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b5504-92a0-4b4f-88cd-6ac0318e85b2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.498290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b5504-92a0-4b4f-88cd-6ac0318e85b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:39.515654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 829/1681] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:39 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:39.517703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:39.647061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b86b5504-92a0-4b4f-88cd-6ac0318e85b2 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:39.648641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b86b5504-92a0-4b4f-88cd-6ac0318e85b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:39.672667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e3ad9-3c3a-4a57-b492-981cca272dc2 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:39.681927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:39.683477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.684447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.684974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:39.685857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:39.689144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:39.725596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 830/1682] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:39 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:26:39.727127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:39.739890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:39.811512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 853/1683] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:26:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:39.815371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b5504-92a0-4b4f-88cd-6ac0318e85b2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:39.854765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b5504-92a0-4b4f-88cd-6ac0318e85b2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:39.863897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:39.865062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.865592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.866087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:39.867059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:39.869451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:39.904444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 854/1684] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:39.910659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:39.928470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0ba8-5f0e-4885-8eb9-9f1c930d9c44 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:39.935279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:39.944419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.944419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.944419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] PATH_INFO: `/v3/projects/32025996d54245669606a59c60c78b2c/users/10f4da3050524bfa8b2f34281763c120/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:39.952717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:39.972841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3f405-0b46-40f0-9ffd-d4b20278b971 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:39.979110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32025996d54245669606a59c60c78b2c, user_id=10f4da3050524bfa8b2f34281763c120, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:39.988898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:39.993395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:39.995736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:39.997137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:39.998181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:40.035274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 855/1685] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:39 2022] PUT /identity/v3/projects/32025996d54245669606a59c60c78b2c/users/10f4da3050524bfa8b2f34281763c120/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:40.043562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:40.107777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:40.208174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31dcb8-9773-46a5-8218-4e589d10d792 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:40.215186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:40.215862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:40.216167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:40.216463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:40.220585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:40.222605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:40.271534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 856/1686] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:40.275291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:40.279438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:40.324742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:40.342649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:40.364980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 831/1687] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:40.366231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:40.367863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:40.389358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296aa0-50a8-43f7-bfd2-c67db59e3012 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:40.396820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:40.397694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:40.398096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:40.398489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] PATH_INFO: `/v3/projects/bd0ca31db2cb4b4da308a70b6954629c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:40.399955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0ca31db2cb4b4da308a70b6954629c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:40.403135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:40.420295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0eb4f-0ffe-4785-b54b-70b9b2ea8590 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:40.427074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:40.427903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:40.428197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:40.428554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:40.503474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0ca31db2cb4b4da308a70b6954629c', 'request_id': 'req-685260a6-7107-40ff-8090-45bc3bfb34bd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:40.517363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:40.519201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:40.520116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0ca31db2cb4b4da308a70b6954629c', 'request_id': 'req-685260a6-7107-40ff-8090-45bc3bfb34bd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:40.537587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0ca31db2cb4b4da308a70b6954629c', 'request_id': 'req-685260a6-7107-40ff-8090-45bc3bfb34bd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:40.537976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 857/1688] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:26:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:26:40.538420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:40.562920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c237e6-cf71-4eb7-9bfd-7c4802dddb2e service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:40.599015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] The token cache is being invalidate because project bd0ca31db2cb4b4da308a70b6954629c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:40.599362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:40.600337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-685260a6-7107-40ff-8090-45bc3bfb34bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:40.605163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 832/1689] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:40 2022] DELETE /identity/v3/projects/bd0ca31db2cb4b4da308a70b6954629c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:40.618036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e781d4b-9d75-44ab-a484-5e1da899feeb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:40.618838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e781d4b-9d75-44ab-a484-5e1da899feeb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:40.618838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e781d4b-9d75-44ab-a484-5e1da899feeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:40.642938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:40.756574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2e781d4b-9d75-44ab-a484-5e1da899feeb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:40.761009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2e781d4b-9d75-44ab-a484-5e1da899feeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:40.768313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7f55e-b6f5-4ca8-abce-46c38965922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:40.772997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7f55e-b6f5-4ca8-abce-46c38965922f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:40.774295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7f55e-b6f5-4ca8-abce-46c38965922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:40.774573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7f55e-b6f5-4ca8-abce-46c38965922f admin admin] PATH_INFO: `/v3/projects/32025996d54245669606a59c60c78b2c/users/10f4da3050524bfa8b2f34281763c120/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:40.787922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7f55e-b6f5-4ca8-abce-46c38965922f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32025996d54245669606a59c60c78b2c, user_id=10f4da3050524bfa8b2f34281763c120, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:40.810476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7f55e-b6f5-4ca8-abce-46c38965922f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:40.821796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 833/1690] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:40.823533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288017-20bf-4c04-877d-29fe83a5d198 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:40.826856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288017-20bf-4c04-877d-29fe83a5d198 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:40.827480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288017-20bf-4c04-877d-29fe83a5d198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:40.884908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-27288017-20bf-4c04-877d-29fe83a5d198 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:40.886890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-27288017-20bf-4c04-877d-29fe83a5d198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:40.887493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 858/1691] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:40 2022] PUT /identity/v3/projects/32025996d54245669606a59c60c78b2c/users/10f4da3050524bfa8b2f34281763c120/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:40.894641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84de535-863c-4137-b3c1-4415aeafb048 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:40.898853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84de535-863c-4137-b3c1-4415aeafb048 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:40.899217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84de535-863c-4137-b3c1-4415aeafb048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:41.046512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b84de535-863c-4137-b3c1-4415aeafb048 None None] MFA Rules not processed for user `10f4da3050524bfa8b2f34281763c120`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:41.049760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b84de535-863c-4137-b3c1-4415aeafb048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:41.174508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 834/1692] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:41.179778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27288017-20bf-4c04-877d-29fe83a5d198 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:41.222070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27288017-20bf-4c04-877d-29fe83a5d198 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:41.237481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:41.238382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:41.238779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:41.239958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:41.241103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:41.241422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:41.243292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:41.264012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 835/1693] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:41.265134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:41.273639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:41.310860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:41.321569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 859/1694] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:41.324061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc0223-5066-4423-b05d-071b2aeb634c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:41.328959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84de535-863c-4137-b3c1-4415aeafb048 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:41.367793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:41.370609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84de535-863c-4137-b3c1-4415aeafb048 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:41.380448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:41.380958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:41.381256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] PATH_INFO: `/v3/users/cad7dcbf9662422f9b93a2eea25c3584` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:41.397565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:41.398234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:41.398683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:41.398980 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:41.400097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:41.403294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:41.405066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cad7dcbf9662422f9b93a2eea25c3584)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:41.413044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:41.540688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 860/1695] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:41 2022] POST /identity/v3/projects => generated 403 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:41.554872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:41.565698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:41.661648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cad7dcbf9662422f9b93a2eea25c3584'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:41.669126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cad7dcbf9662422f9b93a2eea25c3584'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:41.752238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cad7dcbf9662422f9b93a2eea25c3584'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:41.796031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cad7dcbf9662422f9b93a2eea25c3584'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:41.798595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cad7dcbf9662422f9b93a2eea25c3584'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:41.849717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 836/1696] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:41 2022] DELETE /identity/v3/users/cad7dcbf9662422f9b93a2eea25c3584 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:41.851627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:41.856017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:41.895123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e603-1706-4cda-93e6-505c1056b3c7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:42.053471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:42.103786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:42.104639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:42.105819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:42.106043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:42.116433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:42.139911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821aee6-ef6b-4f60-a496-51cbd5efbb36 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:42.168989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a586aa5-b2cb-489a-ac7d-207610943bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:42.169661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a586aa5-b2cb-489a-ac7d-207610943bfe admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:42.169959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a586aa5-b2cb-489a-ac7d-207610943bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:42.170244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a586aa5-b2cb-489a-ac7d-207610943bfe admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:42.170941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a586aa5-b2cb-489a-ac7d-207610943bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:42.180489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a586aa5-b2cb-489a-ac7d-207610943bfe admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:42.232251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 837/1697] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:41 2022] POST /identity/v3/users => generated 302 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:42.235412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819368-fdb3-4e35-8ab9-d97f7715e8ff None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:42.235777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819368-fdb3-4e35-8ab9-d97f7715e8ff None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:42.236251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819368-fdb3-4e35-8ab9-d97f7715e8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:42.283220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-31819368-fdb3-4e35-8ab9-d97f7715e8ff None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:42.284632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-31819368-fdb3-4e35-8ab9-d97f7715e8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:42.303882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 838/1698] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:42.306224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31819368-fdb3-4e35-8ab9-d97f7715e8ff None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:42.328031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31819368-fdb3-4e35-8ab9-d97f7715e8ff None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:42.350524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:42.351221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:42.351537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:42.351825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:42.352584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:42.354264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:42.367316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:42.369042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:42.406186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 839/1699] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:42.433404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:42.433831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 861/1700] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 884 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:42.458278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7230589-5f9f-41dd-81d5-2af14c9c1a0a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:42.489475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f60dba-c5bf-4b24-9d91-a94c9da819c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:42.490812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60dba-c5bf-4b24-9d91-a94c9da819c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:42.491109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60dba-c5bf-4b24-9d91-a94c9da819c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:42.491401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60dba-c5bf-4b24-9d91-a94c9da819c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:42.492160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f60dba-c5bf-4b24-9d91-a94c9da819c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:42.497135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f60dba-c5bf-4b24-9d91-a94c9da819c9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:42.548274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:42.560001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 840/1701] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:42.570595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34b38a-7da1-4a59-a141-961b81a450c5 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:42.596264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:42.597040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:42.597332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:42.597620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] PATH_INFO: `/v3/projects/4efcf124b02d4645b0730c75a9ba923a/users/9b9281ec7beb449fb461cfb43fc413a9/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:42.634208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4efcf124b02d4645b0730c75a9ba923a, user_id=9b9281ec7beb449fb461cfb43fc413a9, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:42.637122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:42.703003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 862/1702] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:42 2022] PUT /identity/v3/projects/4efcf124b02d4645b0730c75a9ba923a/users/9b9281ec7beb449fb461cfb43fc413a9/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:42.707098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbeb8f-8af8-45af-933c-84677efe99e5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:42.707394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbeb8f-8af8-45af-933c-84677efe99e5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:42.707686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbeb8f-8af8-45af-933c-84677efe99e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:42.832489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-27fbeb8f-8af8-45af-933c-84677efe99e5 None None] MFA Rules not processed for user `9b9281ec7beb449fb461cfb43fc413a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:42.834235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-27fbeb8f-8af8-45af-933c-84677efe99e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:42.846946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:42.848972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:42.998200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:43.092869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c88756-72bb-4af7-8019-224e35d135cd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:43.107618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:43.108587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:43.109055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:43.109511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:43.136696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 841/1703] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:43.138088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbeb8f-8af8-45af-933c-84677efe99e5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:43.311680 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:43.313338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:43.326370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbeb8f-8af8-45af-933c-84677efe99e5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:43.343431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:43.343898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:43.344958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:43.345333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:43.346223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:43.348201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:43.350957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 863/1704] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:43.351860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:43.398119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:43.431003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 842/1705] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:43 2022] POST /identity/v3/projects => generated 405 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:43.431707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:43.579565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9929542-7377-49ce-8271-e86e596ec8a2 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:43.636101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd0d4-2db4-4972-ad08-c36f4513fb4e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:43.658973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:43.660907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:43.661365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:43.661814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:43.662795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:43.665313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:43.702366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 843/1706] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:43 2022] POST /identity/v3/users => generated 309 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:43.706582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:43.751641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f27827-67e6-4040-a5ef-c46092497edb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:43.765974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650c8b3-b6a7-45d7-ad08-83251a1976fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:43.771003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650c8b3-b6a7-45d7-ad08-83251a1976fb admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:43.771003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650c8b3-b6a7-45d7-ad08-83251a1976fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:43.771003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650c8b3-b6a7-45d7-ad08-83251a1976fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:43.774664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650c8b3-b6a7-45d7-ad08-83251a1976fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:43.782454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650c8b3-b6a7-45d7-ad08-83251a1976fb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:43.784201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4da3050524bfa8b2f34281763c120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32025996d54245669606a59c60c78b2c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:43.785076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:43.785507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:43.787053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:43.787816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:43.788138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:43.798971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:43.820873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 844/1707] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:43.823056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48fbf8-f2d5-409d-8592-392f14850fe8 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:43.823181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 864/1708] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:43.823873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48fbf8-f2d5-409d-8592-392f14850fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:43.824002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:43.824621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48fbf8-f2d5-409d-8592-392f14850fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:43.843333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0e925-c038-4dc6-b7b4-a0008328deba tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:43.875321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:43.877417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:43.877963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:43.888346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] PATH_INFO: `/v3/projects/2ab0dd66000543d3aec9372c62749b84/users/07b8100a4cd34558add944c7d8fbb1ee/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:43.891989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-bb48fbf8-f2d5-409d-8592-392f14850fe8 None None] MFA Rules not processed for user `edf0a00538cf4e10b3bf266f48ea8cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:43.893816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bb48fbf8-f2d5-409d-8592-392f14850fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:43.915221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab0dd66000543d3aec9372c62749b84, user_id=07b8100a4cd34558add944c7d8fbb1ee, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:43.919090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:43.939963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 865/1709] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:43 2022] PUT /identity/v3/projects/2ab0dd66000543d3aec9372c62749b84/users/07b8100a4cd34558add944c7d8fbb1ee/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:43.941041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:43.997746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:44.098723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 845/1710] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jul 29 06:26:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:44.100224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48fbf8-f2d5-409d-8592-392f14850fe8 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:44.114874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6d7fa-fbe0-4175-805d-c53a842d6684 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:44.126370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4da3050524bfa8b2f34281763c120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32025996d54245669606a59c60c78b2c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:44.127048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:44.127340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:44.133121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48fbf8-f2d5-409d-8592-392f14850fe8 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:44.139122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:44.143149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:44.151364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:44.231099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 866/1711] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:43 2022] POST /identity/v3/projects => generated 354 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:44.232266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:44.319206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:44.373288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:44.374028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:44.374812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:44.374812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:44.375478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:44.377300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:44.433426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 846/1712] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:44.435829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:44.473851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c31d2f-a742-4aef-988c-bb1c84ba56ef admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:44.474148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2af977-1e7e-4173-a6ab-898db5591a52 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:44.482044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:44.497472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:44.497776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:44.498110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:44.533084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:44.535024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:44.574234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 867/1713] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:44.575543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:44.588481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:44.684902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4da3050524bfa8b2f34281763c120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32025996d54245669606a59c60c78b2c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:44.685843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:44.686250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:44.686643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:44.687548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:44.688991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:44.690955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:44.777702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 847/1714] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:44.777702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:44.786226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd051ac6-9650-4d4e-aa29-a7a1cf7fe4ce service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:44.806563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:44.807826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:44.808352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:44.808889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] PATH_INFO: `/v3/projects/027ba2fe28384d3789886b03b4e66c06` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:44.810883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=027ba2fe28384d3789886b03b4e66c06)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:44.817804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:44.817978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441608b8-e080-47a8-b16d-69b2e0d27e2c tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:44.830413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2555a7f-7c4a-42a4-a7d2-e0716ec0cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:44.831052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2555a7f-7c4a-42a4-a7d2-e0716ec0cc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:44.831418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2555a7f-7c4a-42a4-a7d2-e0716ec0cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:44.831697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2555a7f-7c4a-42a4-a7d2-e0716ec0cc67 admin admin] PATH_INFO: `/v3/projects/2ab0dd66000543d3aec9372c62749b84/users/07b8100a4cd34558add944c7d8fbb1ee/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:44.838205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2555a7f-7c4a-42a4-a7d2-e0716ec0cc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab0dd66000543d3aec9372c62749b84, user_id=07b8100a4cd34558add944c7d8fbb1ee, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:44.841363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2555a7f-7c4a-42a4-a7d2-e0716ec0cc67 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:44.890796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 848/1715] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:44 2022] PUT /identity/v3/projects/2ab0dd66000543d3aec9372c62749b84/users/07b8100a4cd34558add944c7d8fbb1ee/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:44.903137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d66a0-f001-47ec-a9c6-aced10d75b93 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:44.903435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d66a0-f001-47ec-a9c6-aced10d75b93 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:44.903819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d66a0-f001-47ec-a9c6-aced10d75b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:45.031035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-082d66a0-f001-47ec-a9c6-aced10d75b93 None None] MFA Rules not processed for user `07b8100a4cd34558add944c7d8fbb1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:45.036671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-082d66a0-f001-47ec-a9c6-aced10d75b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:45.053894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027ba2fe28384d3789886b03b4e66c06', 'request_id': 'req-ba0c4133-353a-403e-8fb4-81a00bada2e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.077855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027ba2fe28384d3789886b03b4e66c06', 'request_id': 'req-ba0c4133-353a-403e-8fb4-81a00bada2e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.096285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027ba2fe28384d3789886b03b4e66c06', 'request_id': 'req-ba0c4133-353a-403e-8fb4-81a00bada2e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.122437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] The token cache is being invalidate because project 027ba2fe28384d3789886b03b4e66c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:45.122803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.126386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.129880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 868/1716] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:44 2022] DELETE /identity/v3/projects/027ba2fe28384d3789886b03b4e66c06 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:45.132452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:45.134758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:45.171877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:45.232877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 849/1717] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:44 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:45.234893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:45.235597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:45.236096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:45.299174 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:45.300352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:45.301429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:45.345639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4133-353a-403e-8fb4-81a00bada2e7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:45.359321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4da3050524bfa8b2f34281763c120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32025996d54245669606a59c60c78b2c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:45.361693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:45.363486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:45.363812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] PATH_INFO: `/v3/projects/80b5af618cf8467b836f5e2b94ac1f3a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:45.367806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorizing `identity:delete_project(project_id=80b5af618cf8467b836f5e2b94ac1f3a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:45.374319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:45.497119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 850/1718] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:45.499810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:45.510092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:45.524591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b5af618cf8467b836f5e2b94ac1f3a', 'request_id': 'req-65d765c0-e82a-41ce-8d82-0eb66956f2a1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.544904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b5af618cf8467b836f5e2b94ac1f3a', 'request_id': 'req-65d765c0-e82a-41ce-8d82-0eb66956f2a1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.555781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b5af618cf8467b836f5e2b94ac1f3a', 'request_id': 'req-65d765c0-e82a-41ce-8d82-0eb66956f2a1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.574408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] The token cache is being invalidate because project 80b5af618cf8467b836f5e2b94ac1f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:45.574737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.575694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65d765c0-e82a-41ce-8d82-0eb66956f2a1 tempest-VolumeQuotaClassesTest-1046837354 tempest-VolumeQuotaClassesTest-1046837354-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:45.581692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 869/1719] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:45 2022] DELETE /identity/v3/projects/80b5af618cf8467b836f5e2b94ac1f3a => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:45.583725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb469d-032b-4f53-9ca4-5add3aaabda6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:45.583982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb469d-032b-4f53-9ca4-5add3aaabda6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:45.584306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb469d-032b-4f53-9ca4-5add3aaabda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:45.609139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b1cb469d-032b-4f53-9ca4-5add3aaabda6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:45.612500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b1cb469d-032b-4f53-9ca4-5add3aaabda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:45.706226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 870/1720] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:45.709100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb469d-032b-4f53-9ca4-5add3aaabda6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:45.719422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb469d-032b-4f53-9ca4-5add3aaabda6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:45.728415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:45.728415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:45.728415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:45.728415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:45.728959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:45.728959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:45.736875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:45.760826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 871/1721] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:45.761165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c987ad-8ae0-4c7e-9e66-ad27e19e9f41 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:45.761765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:45.773981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc245729-1982-469a-9004-c630f20dc02b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:45.779136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:45.779763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:45.780053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:45.780335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:45.781891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:45.784030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:45.884043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 872/1722] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:45 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:45.886376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:46.086387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:46.087315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:46.087618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:46.087909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:46.095676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:46.101351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:46.164470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc20be-cd87-4b9f-88af-dde050795374 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:46.194053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:46.199845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:46.200909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:46.202716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:46.208115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:46.212582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:46.278028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 873/1723] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:45 2022] POST /identity/v3/users => generated 283 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:46.282863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:46.311074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935c573-d867-454e-b7d6-923f04fc9ba4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:46.322931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:46.329223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:46.339548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:46.340115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:46.340731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:46.341715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:46.343582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:46.358845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 874/1724] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:46.363256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:46.387807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:46.389448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:46.399616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865b5f5-0ddc-429a-a9b2-5c0092eb265f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:46.420224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:46.421167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:46.422998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:46.424723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:46.429656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:46.435821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:46.448152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 851/1725] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:45 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 948 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:46.458372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:46.465155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:46.528575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 875/1726] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:46.529666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:46.542003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee151001-59ea-4ee6-8af0-5ec790dbc748 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:46.556183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c13edf-400e-4f9d-a5cd-6e39feae6891 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:46.567079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300f6d3-30e2-4dc4-88d3-a4477a4528f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:46.569368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300f6d3-30e2-4dc4-88d3-a4477a4528f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:46.569872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300f6d3-30e2-4dc4-88d3-a4477a4528f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:46.570500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300f6d3-30e2-4dc4-88d3-a4477a4528f2 admin admin] PATH_INFO: `/v3/projects/fa964fe7a11141eb80f51ebefc31e9ad/users/88fe93c0f0364cbc90cbb9edf0d1726a/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:46.586225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:46.587044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:46.587425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:46.587748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] PATH_INFO: `/v3/users/76463842343a4332be03b61a48d7e9e3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:46.610025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] RBAC: Authorizing `identity:delete_user(user_id=76463842343a4332be03b61a48d7e9e3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:46.612939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:46.622279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300f6d3-30e2-4dc4-88d3-a4477a4528f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa964fe7a11141eb80f51ebefc31e9ad, user_id=88fe93c0f0364cbc90cbb9edf0d1726a, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:46.633807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300f6d3-30e2-4dc4-88d3-a4477a4528f2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:46.677843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 876/1727] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:46 2022] PUT /identity/v3/projects/fa964fe7a11141eb80f51ebefc31e9ad/users/88fe93c0f0364cbc90cbb9edf0d1726a/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:46.710817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658dcac-be8b-449f-af87-cdde0b38b654 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:46.711384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658dcac-be8b-449f-af87-cdde0b38b654 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:46.713808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658dcac-be8b-449f-af87-cdde0b38b654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:46.830310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76463842343a4332be03b61a48d7e9e3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:46.833883 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76463842343a4332be03b61a48d7e9e3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:46.833969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6658dcac-be8b-449f-af87-cdde0b38b654 None None] MFA Rules not processed for user `88fe93c0f0364cbc90cbb9edf0d1726a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:46.837845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6658dcac-be8b-449f-af87-cdde0b38b654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:46.930746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76463842343a4332be03b61a48d7e9e3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:46.951620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76463842343a4332be03b61a48d7e9e3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:46.954033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76463842343a4332be03b61a48d7e9e3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:46.981533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 852/1728] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:46 2022] DELETE /identity/v3/users/76463842343a4332be03b61a48d7e9e3 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:46.984120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:46.989145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:47.091652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 877/1729] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:47.092568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:47.092917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:47.095321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:47.157688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:47.160037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:47.263203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac255bd5-269c-41a0-a0f7-818d61c033da admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:47.278937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:47.280123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:47.280765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:47.281198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] PATH_INFO: `/v3/users/9b9281ec7beb449fb461cfb43fc413a9` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:47.304265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9281ec7beb449fb461cfb43fc413a9)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:47.308057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:47.468094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 878/1730] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:47.469720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:47.488573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:47.544288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3676a8-c546-4e4a-8420-218d9ce83948 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:47.545026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9281ec7beb449fb461cfb43fc413a9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:47.549213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9281ec7beb449fb461cfb43fc413a9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:47.577586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:47.578301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:47.578595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:47.579064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:47.580139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:47.588117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:47.613010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:47.619646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9281ec7beb449fb461cfb43fc413a9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:47.669681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9281ec7beb449fb461cfb43fc413a9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:47.674691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9281ec7beb449fb461cfb43fc413a9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:47.678718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:47.693271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 879/1731] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:47 2022] POST /identity/v3/projects => generated 367 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:47.694679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:47.697774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:47.743744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 853/1732] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:46 2022] DELETE /identity/v3/users/9b9281ec7beb449fb461cfb43fc413a9 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:47.747056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:47.749379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:47.912475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe1a92-c342-40f3-9dd7-4ca8ff41cd2e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:47.925249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:47.925996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:47.926293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:47.926609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:47.933598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dba66-95ee-4913-9ecb-59820505ea9c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:47.936445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:48.128443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:48.130138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:48.130138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:48.130138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] PATH_INFO: `/v3/projects/86746df3894446d6a89a199b7962187c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:48.132257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86746df3894446d6a89a199b7962187c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:48.135051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:48.144489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:48.146100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:48.183888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 854/1733] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:48.184710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:48.190692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:48.223426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159b4a3-118c-4d05-93dd-7acfe65b6e28 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:48.238682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41643698-2143-472a-8c4e-831508214140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:48.239756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41643698-2143-472a-8c4e-831508214140 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:48.240261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41643698-2143-472a-8c4e-831508214140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:48.240717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41643698-2143-472a-8c4e-831508214140 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:48.241755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41643698-2143-472a-8c4e-831508214140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:48.246315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41643698-2143-472a-8c4e-831508214140 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:48.287949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86746df3894446d6a89a199b7962187c', 'request_id': 'req-1eddef52-7f43-4889-b0a7-f007d6347d1c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.302607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 855/1734] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:48 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:48.305388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86746df3894446d6a89a199b7962187c', 'request_id': 'req-1eddef52-7f43-4889-b0a7-f007d6347d1c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.306999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:48.307503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:48.309310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:48.317067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86746df3894446d6a89a199b7962187c', 'request_id': 'req-1eddef52-7f43-4889-b0a7-f007d6347d1c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.350900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] The token cache is being invalidate because project 86746df3894446d6a89a199b7962187c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:48.351387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.352565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.357133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 880/1735] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:47 2022] DELETE /identity/v3/projects/86746df3894446d6a89a199b7962187c => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:48.364235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:48.369387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:48.374194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:48.379277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:48.551189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:48.557324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 856/1736] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:48.558601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:48.560146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:48.578762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf1e6-5aaf-432d-9fc9-ba4a5369b057 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:48.585805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:48.586598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:48.586995 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:48.587385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] PATH_INFO: `/v3/users/4b90be14a70a49df856f4d22e4200f0c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:48.588893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b90be14a70a49df856f4d22e4200f0c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:48.591023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:48.602477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:48.640298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddef52-7f43-4889-b0a7-f007d6347d1c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:48.654675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517241e-3668-493d-8b43-edc9a34e1a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:48.654675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517241e-3668-493d-8b43-edc9a34e1a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:48.654675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517241e-3668-493d-8b43-edc9a34e1a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:48.654675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517241e-3668-493d-8b43-edc9a34e1a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:48.655022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517241e-3668-493d-8b43-edc9a34e1a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:48.657567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517241e-3668-493d-8b43-edc9a34e1a74 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:48.690367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 881/1737] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:48.700964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:48.701338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:48.701614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:48.725140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b90be14a70a49df856f4d22e4200f0c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.727876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b90be14a70a49df856f4d22e4200f0c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.772802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b90be14a70a49df856f4d22e4200f0c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.785687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] MFA Rules not processed for user `217b321936224e2ba928b983c4e40017`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:48.785687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:48.801788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b90be14a70a49df856f4d22e4200f0c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.805291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b90be14a70a49df856f4d22e4200f0c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:48.839224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 857/1738] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:48 2022] DELETE /identity/v3/users/4b90be14a70a49df856f4d22e4200f0c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:48.839224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:48.845037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:49.026794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f09ec8-d4c9-464a-8dd1-8dbb111f0570 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:49.036182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 882/1739] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:49.038768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:49.042369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:49.049375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:49.050224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:49.050518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:49.050802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:49.063444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:49.086672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832971a-43e0-453f-b956-89205acbc6fa None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:49.269011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:49.270221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:49.270656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:49.271087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] PATH_INFO: `/v3/projects/7eadb7f4bd0d4b5385100286701a126b/users/42c2be61a0194ef7bf4903650e758341/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:49.291687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eadb7f4bd0d4b5385100286701a126b, user_id=42c2be61a0194ef7bf4903650e758341, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:49.295012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:49.295771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:49.299048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:49.319588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 883/1740] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:49 2022] PUT /identity/v3/projects/7eadb7f4bd0d4b5385100286701a126b/users/42c2be61a0194ef7bf4903650e758341/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:49.322003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:49.324245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:49.335901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 858/1741] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:49.339902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:49.372872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125d305-02cf-4126-93c2-1368913e45af service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:49.452175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:49.509404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:49.553104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dfcc2-b3c2-4546-8d3f-d2fd6ce48b7b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:49.571472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:49.574844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:49.575222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:49.576525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:49.617598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:49.619488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:49.620817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:49.621157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:49.622064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:49.623923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:49.652043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 859/1742] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:49.671889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:49.700115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c39025-aa53-43b2-afbf-828b56638783 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:49.708278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bb28c-93db-438b-99fb-cce57e5aa2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:49.712358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bb28c-93db-438b-99fb-cce57e5aa2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:49.713164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bb28c-93db-438b-99fb-cce57e5aa2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:49.713584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bb28c-93db-438b-99fb-cce57e5aa2b7 admin admin] PATH_INFO: `/v3/projects/7eadb7f4bd0d4b5385100286701a126b/users/42c2be61a0194ef7bf4903650e758341/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:49.716501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bb28c-93db-438b-99fb-cce57e5aa2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eadb7f4bd0d4b5385100286701a126b, user_id=42c2be61a0194ef7bf4903650e758341, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:49.740342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bb28c-93db-438b-99fb-cce57e5aa2b7 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:49.783532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 860/1743] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:49 2022] PUT /identity/v3/projects/7eadb7f4bd0d4b5385100286701a126b/users/42c2be61a0194ef7bf4903650e758341/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:49.798187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97821d52-551d-4a11-b4b5-8a8e9abcda55 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:49.798484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97821d52-551d-4a11-b4b5-8a8e9abcda55 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:49.798770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97821d52-551d-4a11-b4b5-8a8e9abcda55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:49.830233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:49.834125 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:49.864413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-97821d52-551d-4a11-b4b5-8a8e9abcda55 None None] MFA Rules not processed for user `42c2be61a0194ef7bf4903650e758341`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:49.867404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-97821d52-551d-4a11-b4b5-8a8e9abcda55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:49.887334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 884/1744] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:50.063813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 861/1745] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:50.079626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:50.081750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:50.283281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb411fa8-4725-47bb-ba8c-32b5c891a07d service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:50.435966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b069ff3-4ec6-43d4-8217-e4777e5c6be1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:50.436848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b069ff3-4ec6-43d4-8217-e4777e5c6be1 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:50.442848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b069ff3-4ec6-43d4-8217-e4777e5c6be1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:50.442848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b069ff3-4ec6-43d4-8217-e4777e5c6be1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:50.478275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b069ff3-4ec6-43d4-8217-e4777e5c6be1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:50.480166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b069ff3-4ec6-43d4-8217-e4777e5c6be1 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:50.509938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 885/1746] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:50.863086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97821d52-551d-4a11-b4b5-8a8e9abcda55 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:50.876157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97821d52-551d-4a11-b4b5-8a8e9abcda55 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:50.900006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadca41-f502-4f73-bdf5-c761361ae058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:50.904017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadca41-f502-4f73-bdf5-c761361ae058 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:50.904337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadca41-f502-4f73-bdf5-c761361ae058 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:50.904647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadca41-f502-4f73-bdf5-c761361ae058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:50.915159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-efadca41-f502-4f73-bdf5-c761361ae058 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:51.181459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadca41-f502-4f73-bdf5-c761361ae058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:51.184304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadca41-f502-4f73-bdf5-c761361ae058 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:51.224823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 862/1747] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:51.428632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9313d-c324-4292-a69d-67770115d603 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:51.429330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9313d-c324-4292-a69d-67770115d603 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:51.430569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9313d-c324-4292-a69d-67770115d603 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:51.438818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 886/1748] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:26:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:51.738465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a754469-7396-44db-9262-c03e5e106f93 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:51.738793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a754469-7396-44db-9262-c03e5e106f93 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:51.739044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a754469-7396-44db-9262-c03e5e106f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:51.786621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-6a754469-7396-44db-9262-c03e5e106f93 None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:51.788378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6a754469-7396-44db-9262-c03e5e106f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:51.914996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:52.013323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 863/1749] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:26:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:52.477534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9313d-c324-4292-a69d-67770115d603 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:52.479448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2bb9313d-c324-4292-a69d-67770115d603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:52.489741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a754469-7396-44db-9262-c03e5e106f93 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:52.527473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a754469-7396-44db-9262-c03e5e106f93 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:52.551866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfb2ff-3b68-4f8e-8f24-fac0aefbe047 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:52.552838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfb2ff-3b68-4f8e-8f24-fac0aefbe047 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:52.553272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfb2ff-3b68-4f8e-8f24-fac0aefbe047 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:52.553710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfb2ff-3b68-4f8e-8f24-fac0aefbe047 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:52.572823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfb2ff-3b68-4f8e-8f24-fac0aefbe047 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:52.574812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfb2ff-3b68-4f8e-8f24-fac0aefbe047 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:52.622033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 864/1750] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:52.739497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:52.810387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9313d-c324-4292-a69d-67770115d603 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:52.819506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:52.820196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:52.820485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:52.820809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] PATH_INFO: `/v3/projects/fa36c24d35ff4224bbaee304e355de75` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:52.831076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa36c24d35ff4224bbaee304e355de75)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:52.833620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:52.937463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa36c24d35ff4224bbaee304e355de75', 'request_id': 'req-854565a0-c5b7-4dbe-9f08-fdecc11f722f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:52.961344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa36c24d35ff4224bbaee304e355de75', 'request_id': 'req-854565a0-c5b7-4dbe-9f08-fdecc11f722f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:52.976590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa36c24d35ff4224bbaee304e355de75', 'request_id': 'req-854565a0-c5b7-4dbe-9f08-fdecc11f722f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.010498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] The token cache is being invalidate because project fa36c24d35ff4224bbaee304e355de75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:53.011417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.014552 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-854565a0-c5b7-4dbe-9f08-fdecc11f722f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.016665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 887/1751] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:52 2022] DELETE /identity/v3/projects/fa36c24d35ff4224bbaee304e355de75 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:53.021195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:53.022420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:53.022932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:53.074616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:53.076088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:53.125227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583b5d-b9de-4a52-aae1-102ea81ced78 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:53.125784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583b5d-b9de-4a52-aae1-102ea81ced78 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:53.126303 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583b5d-b9de-4a52-aae1-102ea81ced78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:53.198942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0d583b5d-b9de-4a52-aae1-102ea81ced78 None None] MFA Rules not processed for user `e1b9767f26254a6cba945f4cae562b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:53.201457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0d583b5d-b9de-4a52-aae1-102ea81ced78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:53.356635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 865/1752] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:53.356635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:53.356635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:53.411006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911c23b-154e-4ffd-bb28-42aa3a46317f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:53.426439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 888/1753] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:53.436521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8b1fd-5cdb-40cd-bd57-e2a64cc54df0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:53.436835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8b1fd-5cdb-40cd-bd57-e2a64cc54df0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:53.440930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8b1fd-5cdb-40cd-bd57-e2a64cc54df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:53.476003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:53.476723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:53.477020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:53.477307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] PATH_INFO: `/v3/projects/51197c74305348a48f4b8d4e5f3ce581` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:53.485992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51197c74305348a48f4b8d4e5f3ce581)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:53.488003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:53.508147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-78a8b1fd-5cdb-40cd-bd57-e2a64cc54df0 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:53.524058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-78a8b1fd-5cdb-40cd-bd57-e2a64cc54df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:53.559663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 889/1754] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:53.563729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8b1fd-5cdb-40cd-bd57-e2a64cc54df0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:53.606283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8b1fd-5cdb-40cd-bd57-e2a64cc54df0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:53.627211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:53.628073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:53.628338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:53.628652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:53.629546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:53.629865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:53.636427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:53.669770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 890/1755] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:26:53.675613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:53.697583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cd0f0-9f28-42de-a8ce-a06023da8671 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:53.705681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51197c74305348a48f4b8d4e5f3ce581', 'request_id': 'req-15e857fa-5a84-4c19-85d1-089bd96e9a1e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.732098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51197c74305348a48f4b8d4e5f3ce581', 'request_id': 'req-15e857fa-5a84-4c19-85d1-089bd96e9a1e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.751191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51197c74305348a48f4b8d4e5f3ce581', 'request_id': 'req-15e857fa-5a84-4c19-85d1-089bd96e9a1e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.774292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] The token cache is being invalidate because project 51197c74305348a48f4b8d4e5f3ce581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:53.774783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.775865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:53.783963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 866/1756] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:53 2022] DELETE /identity/v3/projects/51197c74305348a48f4b8d4e5f3ce581 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:53.785858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:53.789985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:53.932229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:53.959872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:53.967131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:53.967865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:53.968160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:53.968446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:53.969973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:53.972143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:54.057044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e857fa-5a84-4c19-85d1-089bd96e9a1e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:54.079505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:54.088211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:54.090559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:54.091153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] PATH_INFO: `/v3/users/10f4da3050524bfa8b2f34281763c120` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:54.112384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10f4da3050524bfa8b2f34281763c120)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:54.117108 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:54.120141 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 891/1757] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:53 2022] POST /identity/v3/projects => generated 377 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:54.122880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:54.130644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:54.285212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10f4da3050524bfa8b2f34281763c120'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:54.289042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10f4da3050524bfa8b2f34281763c120'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:54.296940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d3df-18fb-4622-b8f4-43ca9b4d4359 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:54.303730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:54.304514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:54.304869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:54.305151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:54.307625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:54.311690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:54.337259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10f4da3050524bfa8b2f34281763c120'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:54.360337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10f4da3050524bfa8b2f34281763c120'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:54.361445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10f4da3050524bfa8b2f34281763c120'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:54.362618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 892/1758] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:54 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:54.366600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:54.368493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:54.388379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 867/1759] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:53 2022] DELETE /identity/v3/users/10f4da3050524bfa8b2f34281763c120 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:26:54.415822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178da155-5c1b-456f-b083-6638e06c61ec admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:54.554521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:54.557093 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:54.557537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:54.558119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:54.559209 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:54.562576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:54.581050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 893/1760] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:54.584973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:54.602505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e415a-f2bf-45ac-8c03-fd2df4cbb5d8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:54.614184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeafae-05f6-474a-8e0e-c436755bd009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:54.614841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeafae-05f6-474a-8e0e-c436755bd009 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:54.615186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeafae-05f6-474a-8e0e-c436755bd009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:54.615470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeafae-05f6-474a-8e0e-c436755bd009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:54.616294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeafae-05f6-474a-8e0e-c436755bd009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:54.618197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeafae-05f6-474a-8e0e-c436755bd009 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:54.642125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 868/1761] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:54.643527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:54.655509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e54b5-6a71-417e-829e-e56b50ca2ed6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:54.662508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:54.664681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:54.665025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:54.665411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] PATH_INFO: `/v3/projects/53a35e5bdf7f4b6bb31cd559a4deab99/users/7bf9f78f08444220bd4ee1ffd0b4db29/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:54.694325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a35e5bdf7f4b6bb31cd559a4deab99, user_id=7bf9f78f08444220bd4ee1ffd0b4db29, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:54.698234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:54.736827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 894/1762] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:54 2022] PUT /identity/v3/projects/53a35e5bdf7f4b6bb31cd559a4deab99/users/7bf9f78f08444220bd4ee1ffd0b4db29/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:54.749359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0566-f083-4e5c-b6bd-2f16539f7f8e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:54.750680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0566-f083-4e5c-b6bd-2f16539f7f8e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:54.751300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0566-f083-4e5c-b6bd-2f16539f7f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:54.846248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e54d0566-f083-4e5c-b6bd-2f16539f7f8e None None] MFA Rules not processed for user `7bf9f78f08444220bd4ee1ffd0b4db29`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:54.857409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e54d0566-f083-4e5c-b6bd-2f16539f7f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:55.038013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 869/1763] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:55.042239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:55.156589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:55.191420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70315c4-0507-42ac-9d32-f1d49f45f54d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:55.205418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:55.206078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:55.206368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:55.206652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:55.207330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:55.209855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:55.308021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 895/1764] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:55 2022] POST /identity/v3/projects => generated 375 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:55.312100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0566-f083-4e5c-b6bd-2f16539f7f8e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:55.337788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0566-f083-4e5c-b6bd-2f16539f7f8e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:55.366026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:55.368868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:55.550030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:55.562670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f362b-9289-4c63-a7a1-202b3dedcf50 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:55.576507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:55.577380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:55.577814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:55.578304 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] PATH_INFO: `/v3/projects/4efcf124b02d4645b0730c75a9ba923a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:55.589505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=4efcf124b02d4645b0730c75a9ba923a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:55.591717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:55.615980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:55.616865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:55.617277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:55.617707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:55.620197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:55.630829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:55.701874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4efcf124b02d4645b0730c75a9ba923a', 'request_id': 'req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:55.713586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 870/1765] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:55 2022] POST /identity/v3/users => generated 294 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:55.715106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:55.721363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4efcf124b02d4645b0730c75a9ba923a', 'request_id': 'req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:55.746155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4efcf124b02d4645b0730c75a9ba923a', 'request_id': 'req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:55.769523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99003edd-544b-4ed7-8571-6f49d4d72817 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:55.777626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] The token cache is being invalidate because project 4efcf124b02d4645b0730c75a9ba923a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:55.778661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:55.781475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:55.816278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 896/1766] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:55 2022] DELETE /identity/v3/projects/4efcf124b02d4645b0730c75a9ba923a => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:55.821914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:55.832516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:55.974032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:55.975269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:55.975799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:55.976288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:55.977752 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:55.992421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:56.012830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 871/1767] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:56.014091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:56.019862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:56.067134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e385e-d3ee-4125-b46e-1845d1083cfe admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:56.074890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f1d00-95a7-4933-970d-487eaddecd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:56.075838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1d00-95a7-4933-970d-487eaddecd29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:56.076422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1d00-95a7-4933-970d-487eaddecd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:56.077235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1d00-95a7-4933-970d-487eaddecd29 admin admin] PATH_INFO: `/v3/projects/acc8a6b1ba4d4993b7d8350058a928b4/users/643505e75e1d40f3b199e96ed49484d9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:56.093124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:56.099553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f1d00-95a7-4933-970d-487eaddecd29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc8a6b1ba4d4993b7d8350058a928b4, user_id=643505e75e1d40f3b199e96ed49484d9, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:56.104725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f1d00-95a7-4933-970d-487eaddecd29 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:56.106130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf10dd-9df3-4ce0-ad42-3216675f5ace admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:56.133782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:56.137767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 872/1768] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:56 2022] PUT /identity/v3/projects/acc8a6b1ba4d4993b7d8350058a928b4/users/643505e75e1d40f3b199e96ed49484d9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:56.143193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601836eb-fd0e-4005-92de-5941feb0286c None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:56.143387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601836eb-fd0e-4005-92de-5941feb0286c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:56.145786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601836eb-fd0e-4005-92de-5941feb0286c None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:56.146665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:56.147541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:56.151879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] PATH_INFO: `/v3/users/07b8100a4cd34558add944c7d8fbb1ee` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:56.155440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b8100a4cd34558add944c7d8fbb1ee)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:56.158773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 873/1769] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 29 06:26:56 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:56.159891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601836eb-fd0e-4005-92de-5941feb0286c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:56.163282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:56.297539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b8100a4cd34558add944c7d8fbb1ee'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:56.320934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b8100a4cd34558add944c7d8fbb1ee'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:56.341143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b8100a4cd34558add944c7d8fbb1ee'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:56.362170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b8100a4cd34558add944c7d8fbb1ee'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:56.365894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de2a7734-28dd-41f5-a709-92459cc21157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b8100a4cd34558add944c7d8fbb1ee'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:56.383310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 897/1770] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:55 2022] DELETE /identity/v3/users/07b8100a4cd34558add944c7d8fbb1ee => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:56.385261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:56.393079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:56.393458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:56.401233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601836eb-fd0e-4005-92de-5941feb0286c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:56.444681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] MFA Rules not processed for user `edf0a00538cf4e10b3bf266f48ea8cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:56.446269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:56.595181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:56.611366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 898/1771] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:26:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:56.637127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88767423-602e-40b4-82df-abf4d44a5721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:56.637831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767423-602e-40b4-82df-abf4d44a5721 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:56.638124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767423-602e-40b4-82df-abf4d44a5721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:56.638406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767423-602e-40b4-82df-abf4d44a5721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:56.639118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88767423-602e-40b4-82df-abf4d44a5721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:56.652084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88767423-602e-40b4-82df-abf4d44a5721 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:56.692516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 874/1772] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:56.696602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:56.698653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:56.751169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86addd-4130-4d36-9ba2-8cf214229c33 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:56.787942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:56.789205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:56.789639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:56.790070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] PATH_INFO: `/v3/projects/acc8a6b1ba4d4993b7d8350058a928b4/users/643505e75e1d40f3b199e96ed49484d9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:56.792480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc8a6b1ba4d4993b7d8350058a928b4, user_id=643505e75e1d40f3b199e96ed49484d9, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:56.795534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:56.851632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 899/1773] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:56 2022] PUT /identity/v3/projects/acc8a6b1ba4d4993b7d8350058a928b4/users/643505e75e1d40f3b199e96ed49484d9/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:56.865601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:56.866646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:56.867011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:56.962348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] MFA Rules not processed for user `643505e75e1d40f3b199e96ed49484d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:56.963997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.107628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:57.110570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.159718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 875/1774] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:57.176627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:57.179125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.288460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3142c-2a82-457f-b8b5-afd684e7089b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:57.295743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d129332-031f-48cf-baf5-0d59dde552bb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:57.298851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:57.299996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.301404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.302077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] PATH_INFO: `/v3/projects/32025996d54245669606a59c60c78b2c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.303921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:57.304587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.305489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.306355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.307025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32025996d54245669606a59c60c78b2c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:57.309803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:57.311617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.339639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:57.420726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32025996d54245669606a59c60c78b2c', 'request_id': 'req-39649c48-7462-47bd-9b17-472f8a2688c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:57.432423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32025996d54245669606a59c60c78b2c', 'request_id': 'req-39649c48-7462-47bd-9b17-472f8a2688c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:57.443903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32025996d54245669606a59c60c78b2c', 'request_id': 'req-39649c48-7462-47bd-9b17-472f8a2688c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:57.466797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:57.468430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:57.474153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] The token cache is being invalidate because project 32025996d54245669606a59c60c78b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:57.476004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:57.479823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-39649c48-7462-47bd-9b17-472f8a2688c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:57.480292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 876/1775] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:26:57.483009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 900/1776] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:57 2022] DELETE /identity/v3/projects/32025996d54245669606a59c60c78b2c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:57.488152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea68c-3258-4297-a7a8-55ba307a8df6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.488718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea68c-3258-4297-a7a8-55ba307a8df6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.489144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea68c-3258-4297-a7a8-55ba307a8df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.494864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:57.496725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.568043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ce2ea68c-3258-4297-a7a8-55ba307a8df6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:57.569589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ce2ea68c-3258-4297-a7a8-55ba307a8df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.571275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:57.639118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4c0e5-062a-44fb-8242-1b63b529d3da service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:57.711181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 901/1777] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:57.715174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b180512-4230-41c8-b0ba-5908d612fead None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.716198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b180512-4230-41c8-b0ba-5908d612fead None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.716810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b180512-4230-41c8-b0ba-5908d612fead None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.722084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 902/1778] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:26:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:26:57.727693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a1245-329e-4e34-8238-068de71685d1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.728880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a1245-329e-4e34-8238-068de71685d1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.729805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a1245-329e-4e34-8238-068de71685d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.754226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0170bec-e34f-404b-8c42-a61b2082d56d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:57.754896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0170bec-e34f-404b-8c42-a61b2082d56d service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.755181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0170bec-e34f-404b-8c42-a61b2082d56d service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.755465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0170bec-e34f-404b-8c42-a61b2082d56d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.762917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c0170bec-e34f-404b-8c42-a61b2082d56d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.787508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2e2a1245-329e-4e34-8238-068de71685d1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:57.790644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0170bec-e34f-404b-8c42-a61b2082d56d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:57.791590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2e2a1245-329e-4e34-8238-068de71685d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.797202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0170bec-e34f-404b-8c42-a61b2082d56d service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:57.814599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 903/1779] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:57.817525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a1245-329e-4e34-8238-068de71685d1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:57.826404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 877/1780] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:26:57.844476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a1245-329e-4e34-8238-068de71685d1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:57.862704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:57.864723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.866086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.867350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.869294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3487-ba66-46f3-a648-d73b6afb4903 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.869588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3487-ba66-46f3-a648-d73b6afb4903 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.870003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3487-ba66-46f3-a648-d73b6afb4903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.872806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:26:57.873458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:57.875620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:57.892010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 904/1781] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:26:57.895947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:57.911897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0df8f-3a38-4ccf-9245-e487eb89e6a0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:57.917724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:57.918711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:57.920298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:57.921031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:57.922149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:57.922812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-159b3487-ba66-46f3-a648-d73b6afb4903 None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:57.924053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-159b3487-ba66-46f3-a648-d73b6afb4903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:57.925385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:57.989648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 905/1782] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:57 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:57.993708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.078690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 878/1783] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:26:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:58.161347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e08c2-4af8-4496-99b5-d5f7ac7a2bcd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:58.174281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:58.175026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:58.175322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:58.175605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:58.177453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:58.181741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:58.204266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:58.230615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3487-ba66-46f3-a648-d73b6afb4903 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.230950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 906/1784] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:57 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:58.244129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3487-ba66-46f3-a648-d73b6afb4903 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:58.277813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:58.278550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:58.278970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:58.279343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:58.280146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:58.284494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:58.295040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.296915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:58.314191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 879/1785] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:58.317439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.340219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184c879-b2c9-4408-ab0c-bfd6f9efbd99 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:58.351201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:58.351495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:58.351783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:58.352062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:58.352793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:58.355306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:58.370856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 880/1786] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:26:58.371915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.387510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabd21b-d85a-4256-87e4-c8f6f7c7a044 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:58.394932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cffd0-e434-41ae-8f7c-d7cccb335718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:58.395900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cffd0-e434-41ae-8f7c-d7cccb335718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:58.396063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cffd0-e434-41ae-8f7c-d7cccb335718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:58.396285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cffd0-e434-41ae-8f7c-d7cccb335718 admin admin] PATH_INFO: `/v3/projects/f1ddb2a4588d4f0c9d46e89f9f8fa2ab/users/cf63365d8824455182f49ee3b3dd614d/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:58.404428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40404-58ad-4106-8a1d-4a29c76462c5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:58.419994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:58.420992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:58.421652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:58.422047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:58.424258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cffd0-e434-41ae-8f7c-d7cccb335718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ddb2a4588d4f0c9d46e89f9f8fa2ab, user_id=cf63365d8824455182f49ee3b3dd614d, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:58.428520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cffd0-e434-41ae-8f7c-d7cccb335718 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:58.435899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:58.439282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:58.450919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 881/1787] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:58 2022] PUT /identity/v3/projects/f1ddb2a4588d4f0c9d46e89f9f8fa2ab/users/cf63365d8824455182f49ee3b3dd614d/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:58.464035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a6843-8d44-4819-8704-b4a5fbf1dbb7 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:58.464712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a6843-8d44-4819-8704-b4a5fbf1dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:58.465269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a6843-8d44-4819-8704-b4a5fbf1dbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:58.467649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 907/1788] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:26:58.498819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c00a6843-8d44-4819-8704-b4a5fbf1dbb7 None None] MFA Rules not processed for user `cf63365d8824455182f49ee3b3dd614d`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:58.500810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c00a6843-8d44-4819-8704-b4a5fbf1dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:58.522021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:58.600707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 882/1789] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:26:58.605831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.632446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36209649-534c-4824-8fa7-7dc2d3f03dfb service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:58.788919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:58.789570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:58.791721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:58.791837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:58.795087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:58.798401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:58.857286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 908/1790] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:26:58 2022] POST /identity/v3/projects => generated 387 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:58.862847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a6843-8d44-4819-8704-b4a5fbf1dbb7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.898106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a6843-8d44-4819-8704-b4a5fbf1dbb7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:58.909216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:58.913479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.030214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.031143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.031509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.032711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.034295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.038802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:59.040757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b0177-6c41-4e3a-a09f-f72e24766971 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:59.047261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.048011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.048371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.048752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] PATH_INFO: `/v3/users/88fe93c0f0364cbc90cbb9edf0d1726a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.050735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fe93c0f0364cbc90cbb9edf0d1726a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.055539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:59.064078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:59.127268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 883/1791] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:26:58 2022] POST /identity/v3/users => generated 300 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:26:59.129594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:59.144991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b173f-4d2d-4216-8caa-4fc9e282209d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:59.152635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.153389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.153773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.154206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.155165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.158638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:59.160597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fe93c0f0364cbc90cbb9edf0d1726a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.195730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 884/1792] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:59.196967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:59.198508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fe93c0f0364cbc90cbb9edf0d1726a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.228854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6b2c6-95a0-4f50-ab84-b12fb40d175c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:59.238944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7c482-ebfc-4b87-9156-107521e009b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.239525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fe93c0f0364cbc90cbb9edf0d1726a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.239831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c482-ebfc-4b87-9156-107521e009b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.240122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c482-ebfc-4b87-9156-107521e009b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.240461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c482-ebfc-4b87-9156-107521e009b6 admin admin] PATH_INFO: `/v3/projects/c00796d0682d4631afb93998ba9f79de/users/7e29d44486b04efd966a1a05444ac9df/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.258670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fe93c0f0364cbc90cbb9edf0d1726a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.259620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fe93c0f0364cbc90cbb9edf0d1726a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.267572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 909/1793] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:26:58 2022] DELETE /identity/v3/users/88fe93c0f0364cbc90cbb9edf0d1726a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:59.268049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7c482-ebfc-4b87-9156-107521e009b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00796d0682d4631afb93998ba9f79de, user_id=7e29d44486b04efd966a1a05444ac9df, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.268753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:59.271058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7c482-ebfc-4b87-9156-107521e009b6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:59.273414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.302383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 885/1794] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:59 2022] PUT /identity/v3/projects/c00796d0682d4631afb93998ba9f79de/users/7e29d44486b04efd966a1a05444ac9df/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:59.303060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.303170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.303456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.326421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:59.328479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.467123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 886/1795] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:26:59.472274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:59.476570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac630de-3d28-4acd-8a05-03b600f3db3d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:59.478277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.486314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.486998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.487290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.487574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] PATH_INFO: `/v3/projects/2ab0dd66000543d3aec9372c62749b84` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.493337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab0dd66000543d3aec9372c62749b84)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.495481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:59.520808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ba37c-0e0c-43a9-bb9c-c5d0f2898cdb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:59.554985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.555650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.555931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.556205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.556942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.558677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:59.597477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 887/1796] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:26:59.598062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:59.605090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.640194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:59.648558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab0dd66000543d3aec9372c62749b84', 'request_id': 'req-afd33d2e-34df-4707-9cd6-8cf102289f98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.674560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab0dd66000543d3aec9372c62749b84', 'request_id': 'req-afd33d2e-34df-4707-9cd6-8cf102289f98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.685071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab0dd66000543d3aec9372c62749b84', 'request_id': 'req-afd33d2e-34df-4707-9cd6-8cf102289f98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.698758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] The token cache is being invalidate because project 2ab0dd66000543d3aec9372c62749b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:26:59.699576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.700693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:26:59.705108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 910/1797] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:26:59 2022] DELETE /identity/v3/projects/2ab0dd66000543d3aec9372c62749b84 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:59.706961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:26:59.709066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.729298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:26:59.792003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713fc04-e435-4f94-a4d3-bd93dc4d3758 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:59.814369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33d2e-34df-4707-9cd6-8cf102289f98 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:26:59.832427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc102c28-5d65-4868-92ed-d46351713d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.832427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc102c28-5d65-4868-92ed-d46351713d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.832427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc102c28-5d65-4868-92ed-d46351713d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.833114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc102c28-5d65-4868-92ed-d46351713d6c admin admin] PATH_INFO: `/v3/projects/c00796d0682d4631afb93998ba9f79de/users/7e29d44486b04efd966a1a05444ac9df/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.838461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc102c28-5d65-4868-92ed-d46351713d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00796d0682d4631afb93998ba9f79de, user_id=7e29d44486b04efd966a1a05444ac9df, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.847562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc102c28-5d65-4868-92ed-d46351713d6c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:26:59.875587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 911/1798] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:26:59 2022] PUT /identity/v3/projects/c00796d0682d4631afb93998ba9f79de/users/7e29d44486b04efd966a1a05444ac9df/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:26:59.877207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c7037-d0c0-44a4-b3b2-6658af7c79e5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.877504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c7037-d0c0-44a4-b3b2-6658af7c79e5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.877789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c7037-d0c0-44a4-b3b2-6658af7c79e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.926300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6b9c7037-d0c0-44a4-b3b2-6658af7c79e5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:26:59.927574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6b9c7037-d0c0-44a4-b3b2-6658af7c79e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.950453 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c39fe7-0194-419f-a8ee-8fe16d23876d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:26:59.951093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c39fe7-0194-419f-a8ee-8fe16d23876d service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:26:59.951422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c39fe7-0194-419f-a8ee-8fe16d23876d service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:26:59.951700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c39fe7-0194-419f-a8ee-8fe16d23876d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:26:59.954221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-25c39fe7-0194-419f-a8ee-8fe16d23876d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:26:59.997580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c39fe7-0194-419f-a8ee-8fe16d23876d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:26:59.999404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c39fe7-0194-419f-a8ee-8fe16d23876d service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.016480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 888/1799] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:00.018794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.019142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.019471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.047425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 912/1800] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:00.050071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1654f-9c06-459f-a152-8b213ca9a121 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.050449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1654f-9c06-459f-a152-8b213ca9a121 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.050737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1654f-9c06-459f-a152-8b213ca9a121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.064488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] MFA Rules not processed for user `7e29d44486b04efd966a1a05444ac9df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:00.066209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:00.101917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-09c1654f-9c06-459f-a152-8b213ca9a121 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:00.103541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-09c1654f-9c06-459f-a152-8b213ca9a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:00.136978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 913/1801] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:00.141561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1654f-9c06-459f-a152-8b213ca9a121 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.153598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1654f-9c06-459f-a152-8b213ca9a121 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.169367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.170204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.170507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.171024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.174435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:00.174898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.182656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.224646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 914/1802] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:00.229773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.260574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 889/1803] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:00.266550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.276650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20783120-811d-4689-84a8-d9f17245a45d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.282843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:00.295715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.298629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.299148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.303894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.305113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.310951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.395984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:00.407609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8639ed5-d353-4f9b-9d3a-31e7dc60e99f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.421333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.424971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.425238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.426234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.442577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 915/1804] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:00 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:00.443925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.451679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.456244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.483698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 890/1805] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:00.605358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372da2e-ff1c-4a62-a26c-b2ef73816f95 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.624499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.626949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.628584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.629069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.632361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.639947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.672660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:00.696809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 916/1806] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:00 2022] POST /identity/v3/users => generated 283 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:00.699952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.740005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.761653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0d140-1966-47a6-906d-4b9f9eacbaf4 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.762568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff41b16-a090-4b38-a8a5-6965d85cac7b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.773897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.774744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.775040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.775324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] PATH_INFO: `/v3/users/cf63365d8824455182f49ee3b3dd614d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.776074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.776737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.777022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.777130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf63365d8824455182f49ee3b3dd614d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.777343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.778044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.779819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.782174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.828837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 891/1807] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:00.831016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.846269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f46b1c-7b82-4c98-941c-674f10804d29 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.852281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.853501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.853795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.854075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.854771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.857334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:00.858716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf63365d8824455182f49ee3b3dd614d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:00.871237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf63365d8824455182f49ee3b3dd614d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:00.878712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 892/1808] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:00.878712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.881184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:00.894740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf63365d8824455182f49ee3b3dd614d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:00.907344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf63365d8824455182f49ee3b3dd614d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:00.908215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55610b-223d-4214-96f9-e4a9a09d5def admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:00.908801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf63365d8824455182f49ee3b3dd614d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:00.916091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:00.917072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:00.917409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:00.918446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] PATH_INFO: `/v3/projects/fa964fe7a11141eb80f51ebefc31e9ad` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:00.919864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa964fe7a11141eb80f51ebefc31e9ad)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:00.920250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 917/1809] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:00 2022] DELETE /identity/v3/users/cf63365d8824455182f49ee3b3dd614d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:00.922177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:00.923894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:00.932099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.009055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa964fe7a11141eb80f51ebefc31e9ad', 'request_id': 'req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.019122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa964fe7a11141eb80f51ebefc31e9ad', 'request_id': 'req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.041380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa964fe7a11141eb80f51ebefc31e9ad', 'request_id': 'req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.056120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] The token cache is being invalidate because project fa964fe7a11141eb80f51ebefc31e9ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:01.056676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.057436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-47bb4d8d-1dad-4583-b1e7-4e46ed09f89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.057805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b1ad3-4fa4-44a9-a20b-6ca6a0059cd5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.059260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 893/1810] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:00 2022] DELETE /identity/v3/projects/fa964fe7a11141eb80f51ebefc31e9ad => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:01.060821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.061191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.061535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.084663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:01.086023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.155549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 894/1811] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:01.157568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.158423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.159046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.159146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.159410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.161030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] PATH_INFO: `/v3/projects/8bcfe752b2ba42488ca2d16fd5ea7fd8/users/2c9124b962e44448a084c227174bf77a/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.172114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c23b1b-9ddb-4c00-b299-40834ded6520 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.177207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.177642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bcfe752b2ba42488ca2d16fd5ea7fd8, user_id=2c9124b962e44448a084c227174bf77a, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.178011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.178370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.178724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] PATH_INFO: `/v3/users/42c2be61a0194ef7bf4903650e758341` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.179990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c2be61a0194ef7bf4903650e758341)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.180702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.182065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.184006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:01.208634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 918/1812] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:00 2022] PUT /identity/v3/projects/8bcfe752b2ba42488ca2d16fd5ea7fd8/users/2c9124b962e44448a084c227174bf77a/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:01.209794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.213548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.251071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c2be61a0194ef7bf4903650e758341'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.251780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c2be61a0194ef7bf4903650e758341'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.252964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:01.266790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c2be61a0194ef7bf4903650e758341'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.273682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c2be61a0194ef7bf4903650e758341'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.274372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-cb4242d8-515b-4da6-a63b-d1bc3893229a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c2be61a0194ef7bf4903650e758341'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:01.279358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e27f5-e6bf-48fc-ac1e-09d2d6ef043b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.283674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc81fb8-3739-4034-ae67-8c1ffdaa937a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.284272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81fb8-3739-4034-ae67-8c1ffdaa937a service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.284576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81fb8-3739-4034-ae67-8c1ffdaa937a service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.284852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81fb8-3739-4034-ae67-8c1ffdaa937a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.286623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6cc81fb8-3739-4034-ae67-8c1ffdaa937a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.287946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 895/1813] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:01 2022] DELETE /identity/v3/users/42c2be61a0194ef7bf4903650e758341 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:01.289381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.289739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.290108 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.321143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] MFA Rules not processed for user `2c9124b962e44448a084c227174bf77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:01.322358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.373509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc81fb8-3739-4034-ae67-8c1ffdaa937a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.375526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc81fb8-3739-4034-ae67-8c1ffdaa937a service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.387379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 919/1814] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:01.388886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3bc5-c78c-4e6e-b5bc-7e5fd96e5556 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.389393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3bc5-c78c-4e6e-b5bc-7e5fd96e5556 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.389975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3bc5-c78c-4e6e-b5bc-7e5fd96e5556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.399325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 896/1815] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:01.408642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.411068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.417862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fffe3bc5-c78c-4e6e-b5bc-7e5fd96e5556 None None] MFA Rules not processed for user `217b321936224e2ba928b983c4e40017`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:01.419211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fffe3bc5-c78c-4e6e-b5bc-7e5fd96e5556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.494657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 920/1816] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:01.501766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146e1c-0100-49c7-b88c-b76c643f54cb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.506924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.507665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.508020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.508369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.509195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.511176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.550343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 897/1817] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:01 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:01.551326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3bc5-c78c-4e6e-b5bc-7e5fd96e5556 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.564908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3bc5-c78c-4e6e-b5bc-7e5fd96e5556 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.624351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:01.637185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.638019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.638408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.638792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.639620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.642485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.672659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 921/1818] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:01 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:01.676777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.691413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0472-3a47-4b9b-8446-01eb22bb3183 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.698512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.699408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.699905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.700396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.702400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.704895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.745293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 898/1819] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:01.746547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.761490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a53f2-b550-4f09-aade-51af22fc96b1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.773315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.774647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.774941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.775224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] PATH_INFO: `/v3/projects/626e322c81af44d9a32deaafe2cc9c16/users/7e9e994e55074dd98f90200725c28743/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.800797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626e322c81af44d9a32deaafe2cc9c16, user_id=7e9e994e55074dd98f90200725c28743, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.803466 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.823047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 922/1820] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:01 2022] PUT /identity/v3/projects/626e322c81af44d9a32deaafe2cc9c16/users/7e9e994e55074dd98f90200725c28743/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:01.827057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.832564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.836483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:01.849649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0a14d-e853-4da0-a80a-f1f08d42cc0c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.886282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:01.926851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebd674-1c34-448c-b08e-948aaefd8e83 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.928711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.929555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.930012 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.930360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.931169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.933182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.933312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.934111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.934781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.935215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] PATH_INFO: `/v3/projects/7eadb7f4bd0d4b5385100286701a126b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.937212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eadb7f4bd0d4b5385100286701a126b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.941370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.944637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 899/1821] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:01.948923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:01.960286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44430fe7-2703-486f-8bc3-b7dbeddf0227 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:01.965389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc3c62-cfad-46f3-824a-5ff9d560814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:01.966271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc3c62-cfad-46f3-824a-5ff9d560814a admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.966419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc3c62-cfad-46f3-824a-5ff9d560814a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.966613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc3c62-cfad-46f3-824a-5ff9d560814a admin admin] PATH_INFO: `/v3/projects/626e322c81af44d9a32deaafe2cc9c16/users/7e9e994e55074dd98f90200725c28743/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:01.969595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc3c62-cfad-46f3-824a-5ff9d560814a admin admin] RBAC: Authorizing `identity:create_grant(project_id=626e322c81af44d9a32deaafe2cc9c16, user_id=7e9e994e55074dd98f90200725c28743, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:01.972639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc3c62-cfad-46f3-824a-5ff9d560814a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:01.986468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 900/1822] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:01 2022] PUT /identity/v3/projects/626e322c81af44d9a32deaafe2cc9c16/users/7e9e994e55074dd98f90200725c28743/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:01.996692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:01.997099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:01.997393 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.024046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eadb7f4bd0d4b5385100286701a126b', 'request_id': 'req-18f6f69c-0634-46ff-8336-20f22d6edc36'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.028088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] MFA Rules not processed for user `7e9e994e55074dd98f90200725c28743`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:02.029776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:02.034037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eadb7f4bd0d4b5385100286701a126b', 'request_id': 'req-18f6f69c-0634-46ff-8336-20f22d6edc36'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.049232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eadb7f4bd0d4b5385100286701a126b', 'request_id': 'req-18f6f69c-0634-46ff-8336-20f22d6edc36'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.063807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] The token cache is being invalidate because project 7eadb7f4bd0d4b5385100286701a126b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:02.064106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.064853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-18f6f69c-0634-46ff-8336-20f22d6edc36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.066928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 923/1823] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:01 2022] DELETE /identity/v3/projects/7eadb7f4bd0d4b5385100286701a126b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:02.077440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:02.077862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:02.078218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.109915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:02.111900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:02.157410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 901/1824] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:01 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:02.158548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:02.160520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:02.211252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:02.244037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 924/1825] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:02.245791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:02.247768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:02.279623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7954390-8f8b-4627-94bb-039c1742178a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:02.302250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:02.302921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:02.303201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:02.303476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] PATH_INFO: `/v3/projects/f1ddb2a4588d4f0c9d46e89f9f8fa2ab` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.304806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ddb2a4588d4f0c9d46e89f9f8fa2ab)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:02.306950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:02.346423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659408c-f74d-4f87-8d74-832fc0ccabcb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:02.351319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e4b57-3bf5-44a4-879f-3285fb1bb652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:02.352136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4b57-3bf5-44a4-879f-3285fb1bb652 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:02.352664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4b57-3bf5-44a4-879f-3285fb1bb652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:02.352948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4b57-3bf5-44a4-879f-3285fb1bb652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.364636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e4b57-3bf5-44a4-879f-3285fb1bb652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:02.366347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e4b57-3bf5-44a4-879f-3285fb1bb652 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:02.368193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:02.385953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ddb2a4588d4f0c9d46e89f9f8fa2ab', 'request_id': 'req-eefc23d9-a80b-4c9c-90b9-a75733b37847'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.393942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 925/1826] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:02.394243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ddb2a4588d4f0c9d46e89f9f8fa2ab', 'request_id': 'req-eefc23d9-a80b-4c9c-90b9-a75733b37847'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.397829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef2745-f9bd-4008-88b7-b578ccfa357f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:02.398258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef2745-f9bd-4008-88b7-b578ccfa357f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:02.398642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef2745-f9bd-4008-88b7-b578ccfa357f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.408385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ddb2a4588d4f0c9d46e89f9f8fa2ab', 'request_id': 'req-eefc23d9-a80b-4c9c-90b9-a75733b37847'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.428072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] The token cache is being invalidate because project f1ddb2a4588d4f0c9d46e89f9f8fa2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:02.428365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.429069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.431399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 902/1827] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:02 2022] DELETE /identity/v3/projects/f1ddb2a4588d4f0c9d46e89f9f8fa2ab => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:02.435163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:02.438588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:02.444769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-10ef2745-f9bd-4008-88b7-b578ccfa357f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:02.446131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-10ef2745-f9bd-4008-88b7-b578ccfa357f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:02.617248 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 926/1828] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:02.623751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef2745-f9bd-4008-88b7-b578ccfa357f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:02.661718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef2745-f9bd-4008-88b7-b578ccfa357f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:02.667778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc23d9-a80b-4c9c-90b9-a75733b37847 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:02.672671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:02.678326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:02.679430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:02.679778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.680706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:02.681023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:02.685040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:02.685182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:02.686731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:02.687888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:02.688451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] PATH_INFO: `/v3/users/7e29d44486b04efd966a1a05444ac9df` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.694474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e29d44486b04efd966a1a05444ac9df)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:02.696965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:02.728067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 927/1829] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:02.732158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:02.760924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cb7f9-e4bf-441b-baf8-59b3b4f934bf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:02.772597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:02.773516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:02.773802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:02.774819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:02.775752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:02.778120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:02.831749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e29d44486b04efd966a1a05444ac9df'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.834378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e29d44486b04efd966a1a05444ac9df'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.875493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e29d44486b04efd966a1a05444ac9df'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.892352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e29d44486b04efd966a1a05444ac9df'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.894164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e29d44486b04efd966a1a05444ac9df'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:02.900098 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 928/1830] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:02 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:02.907667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:02.916947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:02.935982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 903/1831] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:02 2022] DELETE /identity/v3/users/7e29d44486b04efd966a1a05444ac9df => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:02.937855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:02.939780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:03.017825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:03.024748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea832b7b-afe8-4c16-8eb3-18d5b7f81271 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:03.051438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:03.073750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8121db4-7be8-4a91-8a64-fb283899e37d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:03.094039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:03.094860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.095970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.096406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.098849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:03.101490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:03.145420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6230cc6-e424-49b1-a7ff-89ec87c73d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:03.146894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6230cc6-e424-49b1-a7ff-89ec87c73d34 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.148911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6230cc6-e424-49b1-a7ff-89ec87c73d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.150737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6230cc6-e424-49b1-a7ff-89ec87c73d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.156904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c6230cc6-e424-49b1-a7ff-89ec87c73d34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:03.160464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 904/1832] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:02 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:03.163235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:03.194044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc2921-6207-4641-902c-0f9cd42bddf8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:03.206900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:03.208628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.211397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.211621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.212368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:03.216420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:03.247802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 905/1833] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:03.251500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:03.280627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d885a0-cda6-42cd-9a8b-7584b635b864 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:03.290743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:03.291383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.292407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.292754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.293463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:03.303614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:03.368162 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 906/1834] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:03.379251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:03.386497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6230cc6-e424-49b1-a7ff-89ec87c73d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:03.394165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6230cc6-e424-49b1-a7ff-89ec87c73d34 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:03.419909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4cdc9-c8f8-481c-882d-1608336a1d4a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:03.441445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 929/1835] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:03.442693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:03.446729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.447032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.447304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] PATH_INFO: `/v3/projects/e6bf0dc66a5140d1b06caee004fe5107/users/2deb7ed34f6a481ebf66857c6a497190/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.491739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bf0dc66a5140d1b06caee004fe5107, user_id=2deb7ed34f6a481ebf66857c6a497190, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:03.496492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:03.553138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 907/1836] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:03 2022] PUT /identity/v3/projects/e6bf0dc66a5140d1b06caee004fe5107/users/2deb7ed34f6a481ebf66857c6a497190/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:03.565267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.565656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.565923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.583400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:03.585712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:03.661171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] MFA Rules not processed for user `2deb7ed34f6a481ebf66857c6a497190`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:03.667426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:03.709126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b91698-7591-4084-9a47-3652c311a369 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:03.715343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:03.716087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.716454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.716764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] PATH_INFO: `/v3/users/2c9124b962e44448a084c227174bf77a` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.719137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c9124b962e44448a084c227174bf77a)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:03.721810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:03.823322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c9124b962e44448a084c227174bf77a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:03.824267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c9124b962e44448a084c227174bf77a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:03.851360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c9124b962e44448a084c227174bf77a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:03.870978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c9124b962e44448a084c227174bf77a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:03.872071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3da57618-ebcf-449b-abd0-20b7a3c4f23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c9124b962e44448a084c227174bf77a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:03.884130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 930/1837] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:03.885316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:03.893240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:03.941060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 908/1838] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:03 2022] DELETE /identity/v3/users/2c9124b962e44448a084c227174bf77a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:03.949158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:03.949453 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:03.953050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:03.960760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08808bf2-7892-4dfc-8801-1436c2d7fe47 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.002563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:04.003921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.009512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:04.070053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:04.082566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.083463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.083745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.084207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.084905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.088992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.112037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 909/1839] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:04.122537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.124240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.135478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 931/1840] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:03 2022] POST /identity/v3/projects => generated 385 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:04.140897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.213830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6b413-46d6-46cb-a774-f5ca88fd7cb2 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.220786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb54d6-79ff-4419-bc36-0c6147776a93 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.226848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.227585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.227974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.228284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.229037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.231031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.254885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 932/1841] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:04 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:04.255725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.257343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.266871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.267502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.267782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.268056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.269534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535695cb-b4d9-4bc6-aa9f-b2b3e6917eba admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.274154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.274785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.275079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.275366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] PATH_INFO: `/v3/projects/c00796d0682d4631afb93998ba9f79de` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.276531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00796d0682d4631afb93998ba9f79de)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.277187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.278471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.278573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.287956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 910/1842] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:04.288851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.298295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba64396-f4e8-4bd9-be8a-d0109c901a2d service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.303240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.303865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.304129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.304524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.305445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.307733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.320588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 911/1843] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:04.323513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.334715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f6f5b-d49d-4a45-83e9-ed401160be56 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.335565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00796d0682d4631afb93998ba9f79de', 'request_id': 'req-aca9b94c-186a-4225-8613-c6a5001fda97'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.352998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00796d0682d4631afb93998ba9f79de', 'request_id': 'req-aca9b94c-186a-4225-8613-c6a5001fda97'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.366014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00796d0682d4631afb93998ba9f79de', 'request_id': 'req-aca9b94c-186a-4225-8613-c6a5001fda97'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.383785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] The token cache is being invalidate because project c00796d0682d4631afb93998ba9f79de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:04.384363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.385285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aca9b94c-186a-4225-8613-c6a5001fda97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.387484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 933/1844] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:04 2022] DELETE /identity/v3/projects/c00796d0682d4631afb93998ba9f79de => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:04.392206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c68c1-2094-4675-873d-0f696d87a649 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.392505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c68c1-2094-4675-873d-0f696d87a649 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.392830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c68c1-2094-4675-873d-0f696d87a649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.404047 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:04.411812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.412791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.413081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.413354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] PATH_INFO: `/v3/projects/a50cf29ec2d8406b8292060d75b61b8c/users/41fd9fd47c07420aa86aeb2a84dc201b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.418066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0a1c68c1-2094-4675-873d-0f696d87a649 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:04.419565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0a1c68c1-2094-4675-873d-0f696d87a649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.430594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50cf29ec2d8406b8292060d75b61b8c, user_id=41fd9fd47c07420aa86aeb2a84dc201b, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.432190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 934/1845] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:04.434459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.441445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237398-8793-4e43-bcb7-5efd7550d7bc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.441743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237398-8793-4e43-bcb7-5efd7550d7bc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.442029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237398-8793-4e43-bcb7-5efd7550d7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.453390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 912/1846] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:04 2022] PUT /identity/v3/projects/a50cf29ec2d8406b8292060d75b61b8c/users/41fd9fd47c07420aa86aeb2a84dc201b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:04.458166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.459725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.465986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a9237398-8793-4e43-bcb7-5efd7550d7bc None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:04.467352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a9237398-8793-4e43-bcb7-5efd7550d7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.522837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:04.545025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73bbeb-6bc2-48b2-a34a-93577f9ce7b8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.548570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 935/1847] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:04.550807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.551857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.551967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.552254 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.553010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.553197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237398-8793-4e43-bcb7-5efd7550d7bc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.554683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.566221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237398-8793-4e43-bcb7-5efd7550d7bc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.568646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 913/1848] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:04.572587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.574353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.575008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.575294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.575576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.576278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:04.576624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.578843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.584472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978527c-7455-4831-ae21-5ddf46f6d62d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.591170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae8d2b-aa9d-460b-9183-2aeff6f5ad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.592124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae8d2b-aa9d-460b-9183-2aeff6f5ad23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.596911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae8d2b-aa9d-460b-9183-2aeff6f5ad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.597213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae8d2b-aa9d-460b-9183-2aeff6f5ad23 admin admin] PATH_INFO: `/v3/projects/a50cf29ec2d8406b8292060d75b61b8c/users/41fd9fd47c07420aa86aeb2a84dc201b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.599443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae8d2b-aa9d-460b-9183-2aeff6f5ad23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50cf29ec2d8406b8292060d75b61b8c, user_id=41fd9fd47c07420aa86aeb2a84dc201b, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.603025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae8d2b-aa9d-460b-9183-2aeff6f5ad23 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.605991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 936/1849] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:04.610351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.617757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 914/1850] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:04 2022] PUT /identity/v3/projects/a50cf29ec2d8406b8292060d75b61b8c/users/41fd9fd47c07420aa86aeb2a84dc201b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:04.623545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a97f9-87bb-4fa8-9440-7a90ffb5ce53 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.630877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac37e49-34b6-4a54-9653-3524209a6ade None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.631811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac37e49-34b6-4a54-9653-3524209a6ade None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.632303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac37e49-34b6-4a54-9653-3524209a6ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.668856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-aac37e49-34b6-4a54-9653-3524209a6ade None None] MFA Rules not processed for user `41fd9fd47c07420aa86aeb2a84dc201b`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:04.670488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-aac37e49-34b6-4a54-9653-3524209a6ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.691588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.692293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.692696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.693040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.693823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.696028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.727948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 937/1851] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:04 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:04.728832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.730443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.753931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 915/1852] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:04.758075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac37e49-34b6-4a54-9653-3524209a6ade None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.768264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac37e49-34b6-4a54-9653-3524209a6ade None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.798720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390eb0e2-30f1-475f-99dd-59aa808a694d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.803595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.804274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.804587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.804875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] PATH_INFO: `/v3/projects/8bcfe752b2ba42488ca2d16fd5ea7fd8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.806044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bcfe752b2ba42488ca2d16fd5ea7fd8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.807960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.813932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:04.827224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.827936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.828218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.828489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.829176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:04.831121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:04.853139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 916/1853] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:04 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:04.854627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.856797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.865075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bcfe752b2ba42488ca2d16fd5ea7fd8', 'request_id': 'req-215ffb88-bc85-41b3-9868-19f3a1bd64e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.874390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bcfe752b2ba42488ca2d16fd5ea7fd8', 'request_id': 'req-215ffb88-bc85-41b3-9868-19f3a1bd64e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.876036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:04.882114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bcfe752b2ba42488ca2d16fd5ea7fd8', 'request_id': 'req-215ffb88-bc85-41b3-9868-19f3a1bd64e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.897241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] The token cache is being invalidate because project 8bcfe752b2ba42488ca2d16fd5ea7fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:04.897543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.898224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:04.900051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 938/1854] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:04 2022] DELETE /identity/v3/projects/8bcfe752b2ba42488ca2d16fd5ea7fd8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:04.900617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:04.902066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:04.932240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06194e96-19f9-4c9b-996f-ff51a5daaf8b admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.994790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:04.995620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ffb88-bc85-41b3-9868-19f3a1bd64e9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:04.995740 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:04.996125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:04.996506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:04.999212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.000499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.001165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.001448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.001729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.002533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.004137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.020789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 939/1855] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:05.022470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.024505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.040344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbe00-6bfd-4473-bae9-f4f18f13bfe4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.045570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.046220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.046647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.047053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] PATH_INFO: `/v3/users/7e9e994e55074dd98f90200725c28743` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.049256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e9e994e55074dd98f90200725c28743)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.051246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.090745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.092414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.104476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e9e994e55074dd98f90200725c28743'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.106309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 917/1856] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:05.107226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.112273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e9e994e55074dd98f90200725c28743'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.120019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30411d3b-5586-4f1b-b234-4478798f822e service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.127299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffb2bf-8f0d-4b84-867c-e02ce2476b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.128123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb2bf-8f0d-4b84-867c-e02ce2476b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.128593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb2bf-8f0d-4b84-867c-e02ce2476b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.128903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e9e994e55074dd98f90200725c28743'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.129046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb2bf-8f0d-4b84-867c-e02ce2476b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.129901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffb2bf-8f0d-4b84-867c-e02ce2476b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.131801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffb2bf-8f0d-4b84-867c-e02ce2476b9e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.149070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e9e994e55074dd98f90200725c28743'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.150912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e9e994e55074dd98f90200725c28743'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.159867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 940/1857] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:05 2022] DELETE /identity/v3/users/7e9e994e55074dd98f90200725c28743 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:05.166265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 918/1858] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:05.166910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.168604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.246483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:05.263504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b963a-afbe-40e3-aba2-14213c95f0b3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.272855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.274239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.274712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.275143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] PATH_INFO: `/v3/projects/4ed8efdb575341f3bf573e2d13a100c7/users/94a40e737a7d4ae984e086d0493547d7/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.294432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed8efdb575341f3bf573e2d13a100c7, user_id=94a40e737a7d4ae984e086d0493547d7, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.298578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.320626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 941/1859] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:05 2022] PUT /identity/v3/projects/4ed8efdb575341f3bf573e2d13a100c7/users/94a40e737a7d4ae984e086d0493547d7/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:05.330785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c56a70-8290-40ed-8589-fcba8e07c533 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.331076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c56a70-8290-40ed-8589-fcba8e07c533 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.331511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c56a70-8290-40ed-8589-fcba8e07c533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.395574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-31c56a70-8290-40ed-8589-fcba8e07c533 None None] MFA Rules not processed for user `94a40e737a7d4ae984e086d0493547d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:05.397406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-31c56a70-8290-40ed-8589-fcba8e07c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.445597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:05.520208 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.522622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.541316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 919/1860] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:05.544132 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097424c-6352-488d-ac5f-96bd9a5779ce None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.544732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097424c-6352-488d-ac5f-96bd9a5779ce None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.545294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097424c-6352-488d-ac5f-96bd9a5779ce None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.547877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 920/1861] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:27:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:27:05.548651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097424c-6352-488d-ac5f-96bd9a5779ce None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.646241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac7d7c-58ba-43ca-8e83-e284dc9addfb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.662765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.662765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.664851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.665688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097424c-6352-488d-ac5f-96bd9a5779ce None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.665938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] PATH_INFO: `/v3/projects/626e322c81af44d9a32deaafe2cc9c16` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.669505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] RBAC: Authorizing `identity:delete_project(project_id=626e322c81af44d9a32deaafe2cc9c16)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.672034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a242f5-e6c0-4780-9bda-a63d1ddb8040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.673048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.673159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a242f5-e6c0-4780-9bda-a63d1ddb8040 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.673426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a242f5-e6c0-4780-9bda-a63d1ddb8040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.673699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a242f5-e6c0-4780-9bda-a63d1ddb8040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.674369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a242f5-e6c0-4780-9bda-a63d1ddb8040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.677227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a242f5-e6c0-4780-9bda-a63d1ddb8040 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.712170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 921/1862] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:05 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:05.712385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0937fc5-5886-451c-bc5b-54a534b12c0c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.712554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0937fc5-5886-451c-bc5b-54a534b12c0c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.712882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0937fc5-5886-451c-bc5b-54a534b12c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.741652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f0937fc5-5886-451c-bc5b-54a534b12c0c None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:05.742956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f0937fc5-5886-451c-bc5b-54a534b12c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.750244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626e322c81af44d9a32deaafe2cc9c16', 'request_id': 'req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.759699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626e322c81af44d9a32deaafe2cc9c16', 'request_id': 'req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.768507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626e322c81af44d9a32deaafe2cc9c16', 'request_id': 'req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.781493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] The token cache is being invalidate because project 626e322c81af44d9a32deaafe2cc9c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:05.782043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.783490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:05.788453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 942/1863] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:05 2022] DELETE /identity/v3/projects/626e322c81af44d9a32deaafe2cc9c16 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:05.788453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.788453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.841807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 922/1864] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:27:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:05.843233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.843522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.843799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.861630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2b6b-9d96-4fa5-863f-dcb5a553e03f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.864493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:05.865813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.866509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ba968-140c-4fc0-b057-cc97afa68eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.867132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ba968-140c-4fc0-b057-cc97afa68eda admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.867412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ba968-140c-4fc0-b057-cc97afa68eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.867691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ba968-140c-4fc0-b057-cc97afa68eda admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.868454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ba968-140c-4fc0-b057-cc97afa68eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.870524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ba968-140c-4fc0-b057-cc97afa68eda admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.873817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 923/1865] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:05.875299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.876603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.885191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:05.894220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09709264-adcf-4f9a-9cb4-354983bb2014 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.899968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.900632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.900756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 943/1866] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:05 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:05.901116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.901223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.902054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b1af-5f90-4132-97ab-a21db7723fdc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.902378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b1af-5f90-4132-97ab-a21db7723fdc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.902664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b1af-5f90-4132-97ab-a21db7723fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.911318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.912946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.922638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cb74b1af-5f90-4132-97ab-a21db7723fdc None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:05.923842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cb74b1af-5f90-4132-97ab-a21db7723fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:05.923988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 924/1867] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:05.924723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.932927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 944/1868] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:05.935651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd99a1-2e02-4f59-873c-f73ae70d69ea service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.936929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b1af-5f90-4132-97ab-a21db7723fdc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.940774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.940987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.941274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.941675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.942241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.943872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.946529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b1af-5f90-4132-97ab-a21db7723fdc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.952071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.953394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.953682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.954001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.954652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:05.954944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.956251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 925/1869] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:05.960126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.960612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.971647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed9092-ad9e-4497-851f-55b1be3f7f2e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.976893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 945/1870] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:27:05.977018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.977018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.977319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.977600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] PATH_INFO: `/v3/projects/35dae8488fff44099ead9b5e8f875e53/users/023320ef56bd4c8a87302c352cd0dad1/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.977763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:05.988238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337a7bb-74a2-41e3-a0f0-ea781155666e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:05.990301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:05.993333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:05.993955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:05.994242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:05.994524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:05.994633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=35dae8488fff44099ead9b5e8f875e53, user_id=023320ef56bd4c8a87302c352cd0dad1, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.995214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:05.997336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:05.998138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.013095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 926/1871] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:05 2022] PUT /identity/v3/projects/35dae8488fff44099ead9b5e8f875e53/users/023320ef56bd4c8a87302c352cd0dad1/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:06.017990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.027735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 946/1872] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:05 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:06.032276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.098351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b48f38-7070-4de9-aec7-ddd8db046247 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.098610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0348e9-7455-4139-8a5c-9878f6dc0efb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.104292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.104954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.105152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.105645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.105645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.106011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.106204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.106359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.106626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.107292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.108231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.110427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.132007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 927/1873] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:06.133039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.140766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 947/1874] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:06 2022] POST /identity/v3/users => generated 279 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:06.142461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.146539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f4136-19a8-4ec7-aaf3-a113a4c51805 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.154411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9af57-2ea8-4e96-ac26-917518c3d5ce admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.160914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d77188-e803-4a19-89b5-d38a01636fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.161622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77188-e803-4a19-89b5-d38a01636fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.162070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77188-e803-4a19-89b5-d38a01636fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.162365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77188-e803-4a19-89b5-d38a01636fbd admin admin] PATH_INFO: `/v3/projects/35dae8488fff44099ead9b5e8f875e53/users/023320ef56bd4c8a87302c352cd0dad1/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.164394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d77188-e803-4a19-89b5-d38a01636fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=35dae8488fff44099ead9b5e8f875e53, user_id=023320ef56bd4c8a87302c352cd0dad1, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.167074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d77188-e803-4a19-89b5-d38a01636fbd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.168631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.169272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.169562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.169974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.170680 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.172397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.179192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 928/1875] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:06 2022] PUT /identity/v3/projects/35dae8488fff44099ead9b5e8f875e53/users/023320ef56bd4c8a87302c352cd0dad1/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:06.185066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 948/1876] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:06.190414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.190414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.190660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.190781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.213201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d41f8-1d74-4115-8622-7488146f5dd4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.242357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] MFA Rules not processed for user `023320ef56bd4c8a87302c352cd0dad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:06.244092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:06.290386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.290386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.290615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.290794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.291650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.294283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.313617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 949/1877] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:06.317724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.327440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c411c9-17c6-41df-acaa-3561f17d973b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.333347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a84fb-3b34-4ade-b81f-14a962b97e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.334046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a84fb-3b34-4ade-b81f-14a962b97e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.334295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a84fb-3b34-4ade-b81f-14a962b97e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.334575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a84fb-3b34-4ade-b81f-14a962b97e99 admin admin] PATH_INFO: `/v3/projects/8f4b1463eeac42898c4a71b0ff5f20dd/users/5190401ea9e34bf2b61b2b3d49e14905/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.336599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 929/1878] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:06.351907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.353291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:06.358181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a84fb-3b34-4ade-b81f-14a962b97e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4b1463eeac42898c4a71b0ff5f20dd, user_id=5190401ea9e34bf2b61b2b3d49e14905, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.361589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a84fb-3b34-4ade-b81f-14a962b97e99 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.375399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 950/1879] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:06 2022] PUT /identity/v3/projects/8f4b1463eeac42898c4a71b0ff5f20dd/users/5190401ea9e34bf2b61b2b3d49e14905/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:06.385750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9aa71-4abb-40ae-87ec-18158a1e531c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.386040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9aa71-4abb-40ae-87ec-18158a1e531c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.386315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9aa71-4abb-40ae-87ec-18158a1e531c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.395411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:06.413243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222696b-1fa2-424e-8328-e14dbbdb230a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.417746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cea9aa71-4abb-40ae-87ec-18158a1e531c None None] MFA Rules not processed for user `5190401ea9e34bf2b61b2b3d49e14905`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:06.418963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cea9aa71-4abb-40ae-87ec-18158a1e531c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:06.422911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:06.464786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.465454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.465745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.466024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.497174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 951/1880] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:06 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:06.502935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9aa71-4abb-40ae-87ec-18158a1e531c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.537528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.539287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.549199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 930/1881] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:06.573010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9aa71-4abb-40ae-87ec-18158a1e531c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.582215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.583113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.583399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.583678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.584355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.586372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.614503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 952/1882] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:06 2022] POST /identity/v3/projects => generated 355 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:06.618501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.627798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.630108 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfaba7-065a-45da-a7f8-2c171d5fb700 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.683907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d56a2-fd84-4b57-b23f-402a3eaac769 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.687303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.687940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.688057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.688235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.688509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.688630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.688864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.689139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.689358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.691123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:06.691305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.720796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 931/1883] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:06 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:06.724673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.736305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:06.741342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f19a8-1304-4001-8426-0f4d3171fdfc admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.747205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.747956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.748457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.748834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.750481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.752159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.764575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 932/1884] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:06.772757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.774025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.775470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.782379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71133-8875-4463-9daa-56acf005c668 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.785243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 953/1885] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:06.786950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.787659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.787889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.788166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] PATH_INFO: `/v3/projects/ce1676419eb74c5d8f3c30dab6c1a0ae/users/0f1e9d2e1d9e48838326f16d7e7fe5dd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.804072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1676419eb74c5d8f3c30dab6c1a0ae, user_id=0f1e9d2e1d9e48838326f16d7e7fe5dd, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.806746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.823544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 933/1886] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:06 2022] PUT /identity/v3/projects/ce1676419eb74c5d8f3c30dab6c1a0ae/users/0f1e9d2e1d9e48838326f16d7e7fe5dd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:06.828155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.839496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5288799-5c65-4681-b27a-1724abe092e7 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.915254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342abeb-7ae0-42bb-b6cc-09495f370023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.916091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342abeb-7ae0-42bb-b6cc-09495f370023 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.916454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342abeb-7ae0-42bb-b6cc-09495f370023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.916835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342abeb-7ae0-42bb-b6cc-09495f370023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.917883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342abeb-7ae0-42bb-b6cc-09495f370023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.919589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342abeb-7ae0-42bb-b6cc-09495f370023 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.935469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 954/1887] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:06.940597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:06.958430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d0dcb-13a1-4609-8533-267a6f962be3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:06.963452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:06.964498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.965595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:06.965979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] PATH_INFO: `/v3/projects/ce1676419eb74c5d8f3c30dab6c1a0ae/users/0f1e9d2e1d9e48838326f16d7e7fe5dd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:06.968698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1676419eb74c5d8f3c30dab6c1a0ae, user_id=0f1e9d2e1d9e48838326f16d7e7fe5dd, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:06.971411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:06.987852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 934/1888] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:06 2022] PUT /identity/v3/projects/ce1676419eb74c5d8f3c30dab6c1a0ae/users/0f1e9d2e1d9e48838326f16d7e7fe5dd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:06.999373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a74e8c-2b0e-4f21-a82f-633e582b67df None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:06.999777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a74e8c-2b0e-4f21-a82f-633e582b67df None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:07.000335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a74e8c-2b0e-4f21-a82f-633e582b67df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:07.032164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-51a74e8c-2b0e-4f21-a82f-633e582b67df None None] MFA Rules not processed for user `0f1e9d2e1d9e48838326f16d7e7fe5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:07.033422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-51a74e8c-2b0e-4f21-a82f-633e582b67df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:07.100200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:07.106563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:07.113106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 955/1889] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:06 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:07.126043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a74e8c-2b0e-4f21-a82f-633e582b67df None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:07.178842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405274-2b6d-433a-abd9-830cb80e9f8e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:07.184618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a74e8c-2b0e-4f21-a82f-633e582b67df None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:07.185049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:07.187094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:07.187395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:07.187682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] PATH_INFO: `/v3/users/94a40e737a7d4ae984e086d0493547d7` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:07.188473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a0255-6908-49fe-aed9-a0d6439b0df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:07.188928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a40e737a7d4ae984e086d0493547d7)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:07.189151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a0255-6908-49fe-aed9-a0d6439b0df4 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:07.189448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a0255-6908-49fe-aed9-a0d6439b0df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:07.189731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a0255-6908-49fe-aed9-a0d6439b0df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:07.190975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:07.199749 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a0255-6908-49fe-aed9-a0d6439b0df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:07.202136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a0255-6908-49fe-aed9-a0d6439b0df4 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:07.210160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:07.216244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 956/1890] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:07 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:07.260232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a40e737a7d4ae984e086d0493547d7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:07.260870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a40e737a7d4ae984e086d0493547d7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:07.276202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a40e737a7d4ae984e086d0493547d7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:07.299242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a40e737a7d4ae984e086d0493547d7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:07.301436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a40e737a7d4ae984e086d0493547d7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:07.340664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 935/1891] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:07 2022] DELETE /identity/v3/users/94a40e737a7d4ae984e086d0493547d7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:07.341746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:07.342171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:07.342736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:07.344293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:07.345890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:07.370654 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:07.376283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:07.425709 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd392e1-b0af-4f24-96b7-197de0a10c7d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:07.431834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:07.433252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:07.434311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:07.434742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:07.436959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:07.482919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 957/1892] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:07.493118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:07.496655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:07.499087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:07.563352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:07.568814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:07.609984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 936/1893] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:07 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:07.610868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded42d8b-c356-41f1-ab2f-912e3d91a6e2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:07.629396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:07.631893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:07.633581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:07.634842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:07.653058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:07.655408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:07.686316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 958/1894] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:09.139692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:09.142885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:09.175877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd3876-ecfa-406e-b0e1-4d524221f5fb service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:09.191423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:09.191694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:09.192137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:09.192495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] PATH_INFO: `/v3/projects/4ed8efdb575341f3bf573e2d13a100c7` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:09.195188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed8efdb575341f3bf573e2d13a100c7)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:09.199228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:09.278832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed8efdb575341f3bf573e2d13a100c7', 'request_id': 'req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.290515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed8efdb575341f3bf573e2d13a100c7', 'request_id': 'req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.299801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed8efdb575341f3bf573e2d13a100c7', 'request_id': 'req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.313617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] The token cache is being invalidate because project 4ed8efdb575341f3bf573e2d13a100c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:09.313915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.315263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.317139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 937/1895] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:09 2022] DELETE /identity/v3/projects/4ed8efdb575341f3bf573e2d13a100c7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:09.322647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:09.331837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae482f0-4455-4289-9c48-eb2cee5e78c1 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:09.434435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:09.435091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:09.435537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:09.435768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] PATH_INFO: `/v3/users/023320ef56bd4c8a87302c352cd0dad1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:09.437291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=023320ef56bd4c8a87302c352cd0dad1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:09.439350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:09.497602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:09.500941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023320ef56bd4c8a87302c352cd0dad1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.512994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023320ef56bd4c8a87302c352cd0dad1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.538014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023320ef56bd4c8a87302c352cd0dad1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.552798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023320ef56bd4c8a87302c352cd0dad1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.558587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-375bbba1-112b-416e-8e8f-a9e068546e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023320ef56bd4c8a87302c352cd0dad1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:09.569279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 959/1896] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:09 2022] DELETE /identity/v3/users/023320ef56bd4c8a87302c352cd0dad1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:10.040458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:10.042203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:10.066480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a065f9-4ad1-471c-a2d4-f4ed4107ee27 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:10.209306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:10.210704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.211131 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.211549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] PATH_INFO: `/v3/projects/35dae8488fff44099ead9b5e8f875e53` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.212927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35dae8488fff44099ead9b5e8f875e53)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:10.215537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:10.298095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35dae8488fff44099ead9b5e8f875e53', 'request_id': 'req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:10.306051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35dae8488fff44099ead9b5e8f875e53', 'request_id': 'req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:10.319708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35dae8488fff44099ead9b5e8f875e53', 'request_id': 'req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:10.324693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:10.339850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] The token cache is being invalidate because project 35dae8488fff44099ead9b5e8f875e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:10.342244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:10.344678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b91c991f-80cf-4cc0-8794-d0bd3dd59ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:10.349019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 938/1897] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:10 2022] DELETE /identity/v3/projects/35dae8488fff44099ead9b5e8f875e53 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:10.352455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642f2f-a101-4550-8eb5-18ad107a08dd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.353299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642f2f-a101-4550-8eb5-18ad107a08dd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.353747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642f2f-a101-4550-8eb5-18ad107a08dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.383677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3d642f2f-a101-4550-8eb5-18ad107a08dd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:10.385285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d642f2f-a101-4550-8eb5-18ad107a08dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:10.463302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 960/1898] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:10.479880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c530-05f3-4f4e-9b71-675ae2a0c2fe None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.480917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c530-05f3-4f4e-9b71-675ae2a0c2fe None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.481359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c530-05f3-4f4e-9b71-675ae2a0c2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.520683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8cd0c530-05f3-4f4e-9b71-675ae2a0c2fe None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:10.521997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8cd0c530-05f3-4f4e-9b71-675ae2a0c2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:10.531288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 939/1899] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:10.536081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d642f2f-a101-4550-8eb5-18ad107a08dd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:10.547764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d642f2f-a101-4550-8eb5-18ad107a08dd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:10.553511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:10.554594 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.554946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.555296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.556000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:10.556308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:10.557985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:10.571859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 961/1900] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:10.576396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0c530-05f3-4f4e-9b71-675ae2a0c2fe None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:10.587278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0c530-05f3-4f4e-9b71-675ae2a0c2fe None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:10.592514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:10.593443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.593559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.593748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.594590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:10.596519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:10.657873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 940/1901] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:10 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:10.659227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:10.675052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fcc4f-49cd-435c-8c5b-a4bfe9a89031 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:10.759922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:10.760902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.761373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.761894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.762992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:10.765911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:10.788319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 962/1902] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:10 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:10.792677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:10.803014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855878ff-41df-4d40-801d-50a25b28dda6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:10.808475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:10.809475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.809769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.810049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.810744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:10.812924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:10.823889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 941/1903] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:10.828651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:10.839199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ac17c-95e1-4ac9-9f07-65c67668a43a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:10.843801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c1811-5271-46e5-99e6-5a96b07f7411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:10.844419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c1811-5271-46e5-99e6-5a96b07f7411 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.844732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c1811-5271-46e5-99e6-5a96b07f7411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.845011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c1811-5271-46e5-99e6-5a96b07f7411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.845700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c1811-5271-46e5-99e6-5a96b07f7411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:10.847289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c1811-5271-46e5-99e6-5a96b07f7411 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:10.858297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 963/1904] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:10.863380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:10.880702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405fab-5665-4105-93c2-f4e459459a86 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:10.880702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:10.880702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.880702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.881049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] PATH_INFO: `/v3/projects/865414b5436742a39fd016dddd34d556/users/fc1e741febab41f0a8cefedf60752c22/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.895470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=865414b5436742a39fd016dddd34d556, user_id=fc1e741febab41f0a8cefedf60752c22, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:10.898167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:10.911994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 942/1905] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:10 2022] PUT /identity/v3/projects/865414b5436742a39fd016dddd34d556/users/fc1e741febab41f0a8cefedf60752c22/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:10.917166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d980d7-b6ab-470e-b77b-3f2f4fd186d4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:10.917463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d980d7-b6ab-470e-b77b-3f2f4fd186d4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:10.917746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d980d7-b6ab-470e-b77b-3f2f4fd186d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:10.948525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e8d980d7-b6ab-470e-b77b-3f2f4fd186d4 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:10.951304 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8d980d7-b6ab-470e-b77b-3f2f4fd186d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:10.975397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:11.021234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 964/1906] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:11.033051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:11.034884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:11.118949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387169b-4523-4e67-b693-25928734fbdc admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:11.118949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:11.118949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:11.118949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:11.119348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:11.129205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:11.130636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:11.141107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 943/1907] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:11.843072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f297-ee33-41c1-a07c-8b1e71c583b9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:11.843560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f297-ee33-41c1-a07c-8b1e71c583b9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:11.843784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f297-ee33-41c1-a07c-8b1e71c583b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:11.872825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a428f297-ee33-41c1-a07c-8b1e71c583b9 None None] MFA Rules not processed for user `fc1e741febab41f0a8cefedf60752c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:11.874044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a428f297-ee33-41c1-a07c-8b1e71c583b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:11.957325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 965/1908] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:11.962574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:11.973048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60d5c-6da6-49aa-88c5-15185d946e57 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:11.977661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:11.978284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:11.978591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:11.978865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:11.979537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:11.981468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:12.010005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 944/1909] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:11 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:12.014392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f297-ee33-41c1-a07c-8b1e71c583b9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:12.025638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f297-ee33-41c1-a07c-8b1e71c583b9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:12.090466 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:12.091151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:12.091496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:12.092414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:12.092521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:12.094792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:12.120574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 966/1910] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:12 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:12.124996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:12.143046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522e837-5b61-4ee6-b9f6-ba3428b7b2fd admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:12.159470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:12.160118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:12.160410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:12.160717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:12.161413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:12.163002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:12.179985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 945/1911] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:12.181959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:12.193395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a502a38-15fe-44b3-ae4c-fd36d9a2e003 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:12.204058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:12.204808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:12.205796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:12.205796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] PATH_INFO: `/v3/projects/e7093a96fb8d43f9afd0e198be28a713/users/b2c658686da74fafad632e61704bdc22/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:12.226940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7093a96fb8d43f9afd0e198be28a713, user_id=b2c658686da74fafad632e61704bdc22, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:12.229662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:12.250877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 967/1912] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:12 2022] PUT /identity/v3/projects/e7093a96fb8d43f9afd0e198be28a713/users/b2c658686da74fafad632e61704bdc22/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:12.254688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:12.268152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7fff4-9ee1-465a-9fc2-8e8efba00f74 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:12.374892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:12.381909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:12.382710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:12.383326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:12.383814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:12.385457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:12.387447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:12.400817 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 946/1913] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:12.406130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:12.417169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188d9c5-4dc0-44bd-bbee-09073af3a963 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:12.427460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f17897-ed38-49a7-bed8-fcd16b673e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:12.428159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17897-ed38-49a7-bed8-fcd16b673e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:12.428435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17897-ed38-49a7-bed8-fcd16b673e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:12.429396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17897-ed38-49a7-bed8-fcd16b673e02 admin admin] PATH_INFO: `/v3/projects/e7093a96fb8d43f9afd0e198be28a713/users/b2c658686da74fafad632e61704bdc22/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:12.431492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f17897-ed38-49a7-bed8-fcd16b673e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7093a96fb8d43f9afd0e198be28a713, user_id=b2c658686da74fafad632e61704bdc22, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:12.434366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f17897-ed38-49a7-bed8-fcd16b673e02 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:12.451500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 968/1914] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:12 2022] PUT /identity/v3/projects/e7093a96fb8d43f9afd0e198be28a713/users/b2c658686da74fafad632e61704bdc22/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:13.173829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:13.176198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:13.268473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6eef5-4661-442f-837a-ec3bffb773f6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:13.350305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:13.350844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:13.350844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:13.401288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:13.412778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:13.413268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] MFA Rules not processed for user `b2c658686da74fafad632e61704bdc22`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:13.413623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:13.414011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:13.414557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:13.415764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:13.417029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:13.565784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:13.567631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:13.578693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 969/1915] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:13.581369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 947/1916] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:13.758673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:13.761245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:13.798776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600046e0-b938-42d5-a254-3494d3b0e832 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:13.806607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:13.807309 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:13.807942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:13.808327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:13.932408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:13.936327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:13.953957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 970/1917] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:15.844564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:15.857809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e1681-041d-4d9c-b62e-4797763aa001 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:15.864799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b798d4-85c4-4930-a7b5-e507742f5285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:15.865455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b798d4-85c4-4930-a7b5-e507742f5285 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:15.865784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b798d4-85c4-4930-a7b5-e507742f5285 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:15.866317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b798d4-85c4-4930-a7b5-e507742f5285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:15.877550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b798d4-85c4-4930-a7b5-e507742f5285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:15.879176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b798d4-85c4-4930-a7b5-e507742f5285 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:15.900425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 948/1918] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:22.619922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:22.621569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:22.705307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544717c-8b85-47ec-9cc0-4aabcd5a0972 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:22.711608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:22.712714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:22.712714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:22.713219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] PATH_INFO: `/v3/users/2deb7ed34f6a481ebf66857c6a497190` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:22.714160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2deb7ed34f6a481ebf66857c6a497190)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:22.716132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:22.761698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:22.772914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2deb7ed34f6a481ebf66857c6a497190'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:22.782409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2deb7ed34f6a481ebf66857c6a497190'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:22.795027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2deb7ed34f6a481ebf66857c6a497190'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:22.803314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2deb7ed34f6a481ebf66857c6a497190'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:22.803931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2deb7ed34f6a481ebf66857c6a497190'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:22.810156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 971/1919] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:22 2022] DELETE /identity/v3/users/2deb7ed34f6a481ebf66857c6a497190 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:22.814962 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:22.815480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:22.815579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:22.835793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:22.837044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:22.884917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:22.904226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 949/1920] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:22.914841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:22.916382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:22.977731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915eba-783c-427a-acd7-2757b44d6e62 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:22.981423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:22.982055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:22.982341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:22.982629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:22.992683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:22.994441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:23.004804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 972/1921] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:23.283049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:23.284688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:23.298523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdeb07b-f5c2-4ac9-a3e6-7754a738ee32 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:23.303726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:23.304594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:23.304952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:23.305445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] PATH_INFO: `/v3/projects/e6bf0dc66a5140d1b06caee004fe5107` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:23.306748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6bf0dc66a5140d1b06caee004fe5107)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:23.308904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:23.367611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6bf0dc66a5140d1b06caee004fe5107', 'request_id': 'req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.374436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6bf0dc66a5140d1b06caee004fe5107', 'request_id': 'req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.383903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6bf0dc66a5140d1b06caee004fe5107', 'request_id': 'req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.407963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] The token cache is being invalidate because project e6bf0dc66a5140d1b06caee004fe5107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:23.408297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.409008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.410588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 950/1922] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:23 2022] DELETE /identity/v3/projects/e6bf0dc66a5140d1b06caee004fe5107 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:23.414553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:23.425006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19eb69-620c-4c0e-9e3e-d94305127f04 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:23.482168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:23.482825 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:23.483108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:23.483384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] PATH_INFO: `/v3/users/41fd9fd47c07420aa86aeb2a84dc201b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:23.485247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41fd9fd47c07420aa86aeb2a84dc201b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:23.487161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:23.538499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41fd9fd47c07420aa86aeb2a84dc201b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.545770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41fd9fd47c07420aa86aeb2a84dc201b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.547555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:23.560760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41fd9fd47c07420aa86aeb2a84dc201b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.568342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41fd9fd47c07420aa86aeb2a84dc201b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.569022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99b48038-70fd-4a9f-a456-b25a00cd7834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41fd9fd47c07420aa86aeb2a84dc201b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.575530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 973/1923] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:23 2022] DELETE /identity/v3/users/41fd9fd47c07420aa86aeb2a84dc201b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:23.855721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:23.857215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:23.871244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e07ed-585f-4900-96ab-dcbdfa0ed2bc admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:23.930790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:23.931436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:23.931722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:23.932001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] PATH_INFO: `/v3/projects/a50cf29ec2d8406b8292060d75b61b8c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:23.933174 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50cf29ec2d8406b8292060d75b61b8c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:23.935073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:23.982159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50cf29ec2d8406b8292060d75b61b8c', 'request_id': 'req-29f1374c-44cb-4557-842d-a1b902ba9329'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.988965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50cf29ec2d8406b8292060d75b61b8c', 'request_id': 'req-29f1374c-44cb-4557-842d-a1b902ba9329'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:23.996297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50cf29ec2d8406b8292060d75b61b8c', 'request_id': 'req-29f1374c-44cb-4557-842d-a1b902ba9329'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:24.004142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:24.009717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] The token cache is being invalidate because project a50cf29ec2d8406b8292060d75b61b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:24.010279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:24.011338 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29f1374c-44cb-4557-842d-a1b902ba9329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:24.013281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 951/1924] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:23 2022] DELETE /identity/v3/projects/a50cf29ec2d8406b8292060d75b61b8c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:24.019533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374248e-5139-43de-bef1-467b717a5c0f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.020013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374248e-5139-43de-bef1-467b717a5c0f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.020134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374248e-5139-43de-bef1-467b717a5c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.041738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1374248e-5139-43de-bef1-467b717a5c0f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:24.043027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1374248e-5139-43de-bef1-467b717a5c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:24.103612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 974/1925] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:24.112664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddabf6-ca4b-4c6f-ac4a-1b9c9895f371 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.112956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddabf6-ca4b-4c6f-ac4a-1b9c9895f371 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.113241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddabf6-ca4b-4c6f-ac4a-1b9c9895f371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.132498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-99ddabf6-ca4b-4c6f-ac4a-1b9c9895f371 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:24.133728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-99ddabf6-ca4b-4c6f-ac4a-1b9c9895f371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:24.141668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 952/1926] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:24.145960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374248e-5139-43de-bef1-467b717a5c0f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:24.155593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374248e-5139-43de-bef1-467b717a5c0f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:24.160715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:24.161324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.161606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.161881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.162589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:24.162894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:24.164481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:24.176373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 975/1927] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:27:24.180658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddabf6-ca4b-4c6f-ac4a-1b9c9895f371 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:24.190306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddabf6-ca4b-4c6f-ac4a-1b9c9895f371 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:24.194925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:24.195531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.195812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.196088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.196808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:24.198680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:24.226530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 953/1928] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:24 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:24.231214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:24.242304 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c07635-9459-4a88-ba56-ba81b1c08bbd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:24.301632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:24.302175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.302457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.302728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.303373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:24.305316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:24.326018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 976/1929] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:24 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:24.331026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:24.342003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7133926-6539-445e-832e-5d6f1575d8ff admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:24.347310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:24.347922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.348217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.348494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.349215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:24.350802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:24.360639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 954/1930] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:24.364397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:24.374530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df48c6-3b46-4493-92c3-7151d0bb0162 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:24.379130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c681d-0563-4f5e-aa68-f54c81c55822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:24.379755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c681d-0563-4f5e-aa68-f54c81c55822 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.380022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c681d-0563-4f5e-aa68-f54c81c55822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.380303 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c681d-0563-4f5e-aa68-f54c81c55822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.381026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c681d-0563-4f5e-aa68-f54c81c55822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:24.383006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c681d-0563-4f5e-aa68-f54c81c55822 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:24.394164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 977/1931] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:24.398439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:24.409320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b6649-ddd9-4dfe-b10f-08864cc97c97 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:24.413906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:24.414879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.414879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.415165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] PATH_INFO: `/v3/projects/f081895f333e4a4299a7ca4d6c9ffbe7/users/1299be939f52480cad07e9d4773eddb7/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.434347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f081895f333e4a4299a7ca4d6c9ffbe7, user_id=1299be939f52480cad07e9d4773eddb7, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:24.437773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:24.455281 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 955/1932] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:24 2022] PUT /identity/v3/projects/f081895f333e4a4299a7ca4d6c9ffbe7/users/1299be939f52480cad07e9d4773eddb7/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:24.469618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.470002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.470286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.500910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] MFA Rules not processed for user `1299be939f52480cad07e9d4773eddb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:24.502894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:24.534315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:24.576577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 978/1933] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:24.595023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:24.596847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:24.656772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180c528-185f-44fa-957b-340b13b5704d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:24.660449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db2ed1-03e6-4022-932c-8fed4f308dd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:24.661107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db2ed1-03e6-4022-932c-8fed4f308dd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:24.661390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db2ed1-03e6-4022-932c-8fed4f308dd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:24.661667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db2ed1-03e6-4022-932c-8fed4f308dd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:24.670628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db2ed1-03e6-4022-932c-8fed4f308dd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:24.672639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db2ed1-03e6-4022-932c-8fed4f308dd7 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:24.681993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 956/1934] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:26.835184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:26.836711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:26.907279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276078a-99f1-4aff-9a35-500e4d09dffd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:26.912091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:26.913122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:26.913619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:26.914088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] PATH_INFO: `/v3/users/5190401ea9e34bf2b61b2b3d49e14905` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:26.915463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5190401ea9e34bf2b61b2b3d49e14905)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:26.917646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:26.991082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5190401ea9e34bf2b61b2b3d49e14905'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.000147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5190401ea9e34bf2b61b2b3d49e14905'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.013337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5190401ea9e34bf2b61b2b3d49e14905'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.022693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5190401ea9e34bf2b61b2b3d49e14905'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.023480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5190401ea9e34bf2b61b2b3d49e14905'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.036574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 979/1935] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:26 2022] DELETE /identity/v3/users/5190401ea9e34bf2b61b2b3d49e14905 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:27.041534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:27.042370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:27.042895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:27.064043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:27.065545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:27.166014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 957/1936] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:27.177437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:27.180381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:27.264865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:27.272265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e97609-68c9-4a48-a926-abdb463662d3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:27.283149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:27.284239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:27.284531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:27.284887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:27.304651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:27.306317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:27.324760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 980/1937] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:27.752633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:27.754187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:27.771472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22a3d8-b86e-42ab-9368-4e82224cee61 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:27.778141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:27.778882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:27.779236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:27.779582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] PATH_INFO: `/v3/projects/8f4b1463eeac42898c4a71b0ff5f20dd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:27.783160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f4b1463eeac42898c4a71b0ff5f20dd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:27.785488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:27.880478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:27.897502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f4b1463eeac42898c4a71b0ff5f20dd', 'request_id': 'req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.907531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f4b1463eeac42898c4a71b0ff5f20dd', 'request_id': 'req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.918204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f4b1463eeac42898c4a71b0ff5f20dd', 'request_id': 'req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.931145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] The token cache is being invalidate because project 8f4b1463eeac42898c4a71b0ff5f20dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:27.931445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.932263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:27.936603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 958/1938] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:27 2022] DELETE /identity/v3/projects/8f4b1463eeac42898c4a71b0ff5f20dd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:27.938089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:27.949840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb59f-880f-42bf-bfeb-01692e783840 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:28.032842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:28.034025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:28.034345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:28.034908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] PATH_INFO: `/v3/users/0f1e9d2e1d9e48838326f16d7e7fe5dd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:28.036386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1e9d2e1d9e48838326f16d7e7fe5dd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:28.039147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:28.103733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1e9d2e1d9e48838326f16d7e7fe5dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.118569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1e9d2e1d9e48838326f16d7e7fe5dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.160637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1e9d2e1d9e48838326f16d7e7fe5dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.174692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1e9d2e1d9e48838326f16d7e7fe5dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.175979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7a9332e-ba6b-46a7-bd2d-03a9da67d387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1e9d2e1d9e48838326f16d7e7fe5dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.204791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 981/1939] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:27 2022] DELETE /identity/v3/users/0f1e9d2e1d9e48838326f16d7e7fe5dd => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:28.527218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:28.529010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:28.550575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd1eb0-b8bb-409e-bb12-0f237eae83aa admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:28.615256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:28.616291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:28.616620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:28.616883 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] PATH_INFO: `/v3/projects/ce1676419eb74c5d8f3c30dab6c1a0ae` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:28.618073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1676419eb74c5d8f3c30dab6c1a0ae)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:28.619987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:28.670256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1676419eb74c5d8f3c30dab6c1a0ae', 'request_id': 'req-ffed2a3e-a511-401c-91d4-046406d12a72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.677425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1676419eb74c5d8f3c30dab6c1a0ae', 'request_id': 'req-ffed2a3e-a511-401c-91d4-046406d12a72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.686736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1676419eb74c5d8f3c30dab6c1a0ae', 'request_id': 'req-ffed2a3e-a511-401c-91d4-046406d12a72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.701067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] The token cache is being invalidate because project ce1676419eb74c5d8f3c30dab6c1a0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:28.701285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.701959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ffed2a3e-a511-401c-91d4-046406d12a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:28.703647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 959/1940] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:28 2022] DELETE /identity/v3/projects/ce1676419eb74c5d8f3c30dab6c1a0ae => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:28.708917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95c4f2-fdca-4e91-8964-d93db5bc2671 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:28.709211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95c4f2-fdca-4e91-8964-d93db5bc2671 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:28.709498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95c4f2-fdca-4e91-8964-d93db5bc2671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:28.730127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ee95c4f2-fdca-4e91-8964-d93db5bc2671 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:28.731435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ee95c4f2-fdca-4e91-8964-d93db5bc2671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:28.821203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 982/1941] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:28.830563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e398-568e-401d-a65c-2d9f5831999c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:28.830856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e398-568e-401d-a65c-2d9f5831999c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:28.831143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e398-568e-401d-a65c-2d9f5831999c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:28.874136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a033e398-568e-401d-a65c-2d9f5831999c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:28.876394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a033e398-568e-401d-a65c-2d9f5831999c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:28.887468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 960/1942] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:28.892908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95c4f2-fdca-4e91-8964-d93db5bc2671 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:28.904017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95c4f2-fdca-4e91-8964-d93db5bc2671 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:28.909959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:28.910620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:28.910909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:28.911215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:28.911909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:28.912258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:28.913958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:28.922478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:28.928262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 983/1943] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:28.932595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e398-568e-401d-a65c-2d9f5831999c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:28.943130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e398-568e-401d-a65c-2d9f5831999c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:28.947814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:28.948434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:28.948741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:28.949045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:28.949702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:28.951627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:28.984596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 961/1944] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:28 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:28.987683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:28.998991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5867b-2af2-476f-9fdd-3c4cfa97650e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.056840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.057550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.057915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.058284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.059024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.061019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.084260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 984/1945] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:28 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:29.090069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.104618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93857a9d-1a5b-41da-823b-9a69e5a96167 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.109639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.110270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.110551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.110852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.112626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.113232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.124329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 962/1946] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:29.129650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.141097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa44b9-5a1d-41fa-ad3b-c08ee49584d0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.146687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fb59d-e4dc-4604-95d4-54e6101998b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.147343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb59d-e4dc-4604-95d4-54e6101998b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.147628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb59d-e4dc-4604-95d4-54e6101998b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.147905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb59d-e4dc-4604-95d4-54e6101998b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.148704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fb59d-e4dc-4604-95d4-54e6101998b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.150314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fb59d-e4dc-4604-95d4-54e6101998b8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.164653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 985/1947] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:29.169057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.179124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d7a02-b8c8-499d-b3ca-def1af504280 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.184001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.184708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.184997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.185282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] PATH_INFO: `/v3/projects/7b329e7029d64789acfb42526d6741f8/users/ee26289aef2d42e5bb6412df96cbfb72/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.197783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:29.204990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b329e7029d64789acfb42526d6741f8, user_id=ee26289aef2d42e5bb6412df96cbfb72, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.208172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.240671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 963/1948] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:29 2022] PUT /identity/v3/projects/7b329e7029d64789acfb42526d6741f8/users/ee26289aef2d42e5bb6412df96cbfb72/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:29.244481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568fd2f-8dbe-42ac-85a5-1707610b0c2f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.245473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568fd2f-8dbe-42ac-85a5-1707610b0c2f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.245790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568fd2f-8dbe-42ac-85a5-1707610b0c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.307032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3568fd2f-8dbe-42ac-85a5-1707610b0c2f None None] MFA Rules not processed for user `ee26289aef2d42e5bb6412df96cbfb72`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:29.310414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3568fd2f-8dbe-42ac-85a5-1707610b0c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:29.424773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 986/1949] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:29.429681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.521611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dee952-5eef-4dac-9eeb-11a551d94bdb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.526729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.527361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.527933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.527933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.528607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.530561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.560578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 964/1950] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:29 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:29.564288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568fd2f-8dbe-42ac-85a5-1707610b0c2f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.578680 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568fd2f-8dbe-42ac-85a5-1707610b0c2f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.661130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.661788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.662073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.662408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.663023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.665841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.678044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:29.693958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 987/1951] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:29 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:29.698023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.710169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ec2c7-d2be-48c9-9911-9c6b35110c6c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.715808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.716528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.716915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.717276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.718054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.720706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.739789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 965/1952] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:29.743925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.755911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342dec3-bc1a-46a7-96be-3e9e107cceab admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.762069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa0e0a-ac9e-4a3d-8746-dc027e8fefc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.762864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0e0a-ac9e-4a3d-8746-dc027e8fefc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.763160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0e0a-ac9e-4a3d-8746-dc027e8fefc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.763490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0e0a-ac9e-4a3d-8746-dc027e8fefc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.764186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa0e0a-ac9e-4a3d-8746-dc027e8fefc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.766227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa0e0a-ac9e-4a3d-8746-dc027e8fefc2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.779710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 988/1953] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:29.784081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:29.795532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb210e-22b4-457e-8400-3d3132a7721e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:29.801953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:29.802656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.802950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.803229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] PATH_INFO: `/v3/projects/218ad380043e4ceb93d19ec31b2e45d0/users/c1af69dcee0f4ed7803d400a40fb8ff3/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.818693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=218ad380043e4ceb93d19ec31b2e45d0, user_id=c1af69dcee0f4ed7803d400a40fb8ff3, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:29.821441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:29.838903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 966/1954] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:29 2022] PUT /identity/v3/projects/218ad380043e4ceb93d19ec31b2e45d0/users/c1af69dcee0f4ed7803d400a40fb8ff3/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:29.850264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede7551-d100-4e14-826d-c49b4b12354b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:29.850560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede7551-d100-4e14-826d-c49b4b12354b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:29.850847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede7551-d100-4e14-826d-c49b4b12354b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:29.882596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fede7551-d100-4e14-826d-c49b4b12354b None None] MFA Rules not processed for user `c1af69dcee0f4ed7803d400a40fb8ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:29.883857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fede7551-d100-4e14-826d-c49b4b12354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:29.959142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 989/1955] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:29.963188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.027646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc3fa3-5be4-4af1-845b-d709dd7a5118 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.032723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.033357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.033642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.033925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.034593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.036499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.068242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 967/1956] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:29 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:30.069757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede7551-d100-4e14-826d-c49b4b12354b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.082126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede7551-d100-4e14-826d-c49b4b12354b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.145782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.146053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.146380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.146655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.147501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.149563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.174330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 990/1957] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:30 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:30.175471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.185849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f245481-a5d7-49ed-b54e-09818dfd472d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.191426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.192059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.192478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.192782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.193477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.195369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.212007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 968/1958] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:30.213496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.225810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf7cfc-22b0-4118-bb30-d9cd0e02131d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.231077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.231959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.232271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.232964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] PATH_INFO: `/v3/projects/361775ce832442ae9a70df64707f78bf/users/eae637e08cdc428e8e029a53f128d70b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.250514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361775ce832442ae9a70df64707f78bf, user_id=eae637e08cdc428e8e029a53f128d70b, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.253242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.270830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 991/1959] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:30 2022] PUT /identity/v3/projects/361775ce832442ae9a70df64707f78bf/users/eae637e08cdc428e8e029a53f128d70b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:30.274570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.287098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443a9f1-208f-4c0c-82f0-28a576122a45 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.298058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:30.414462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef759401-3439-4d1a-bc67-f17a0fc3fec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.415915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef759401-3439-4d1a-bc67-f17a0fc3fec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.416209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef759401-3439-4d1a-bc67-f17a0fc3fec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.416488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef759401-3439-4d1a-bc67-f17a0fc3fec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.417365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef759401-3439-4d1a-bc67-f17a0fc3fec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.419147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef759401-3439-4d1a-bc67-f17a0fc3fec9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.432624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 969/1960] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:30.437276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.452077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9b274-de93-4ec8-961c-38807d7bb037 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.465193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.466826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.467270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.467792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] PATH_INFO: `/v3/projects/361775ce832442ae9a70df64707f78bf/users/eae637e08cdc428e8e029a53f128d70b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.471100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361775ce832442ae9a70df64707f78bf, user_id=eae637e08cdc428e8e029a53f128d70b, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.474878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.509580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 992/1961] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:30 2022] PUT /identity/v3/projects/361775ce832442ae9a70df64707f78bf/users/eae637e08cdc428e8e029a53f128d70b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:30.521771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.522500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.523589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.587756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] MFA Rules not processed for user `eae637e08cdc428e8e029a53f128d70b`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:30.590034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:30.662193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.664961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:30.700607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 970/1962] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:30.712726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:30.714822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:30.736675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:30.840779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff94c8a-08f9-4a01-b5ab-65a5fd30793f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.847189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfbbde-427b-42e3-8f0d-d3b86deca8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.848241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfbbde-427b-42e3-8f0d-d3b86deca8eb service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.848777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfbbde-427b-42e3-8f0d-d3b86deca8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.849199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfbbde-427b-42e3-8f0d-d3b86deca8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.851697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94bd6f-2a08-4b2a-844d-bd064a377ba9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:30.877891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:30.877891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:30.877891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:30.877891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] PATH_INFO: `/v3/users/7bf9f78f08444220bd4ee1ffd0b4db29` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:30.878480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf9f78f08444220bd4ee1ffd0b4db29)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.878551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfbbde-427b-42e3-8f0d-d3b86deca8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:30.878624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.878691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfbbde-427b-42e3-8f0d-d3b86deca8eb service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:30.891541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 971/1963] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:30.967912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf9f78f08444220bd4ee1ffd0b4db29'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:30.989702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf9f78f08444220bd4ee1ffd0b4db29'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:31.015272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf9f78f08444220bd4ee1ffd0b4db29'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:31.049540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf9f78f08444220bd4ee1ffd0b4db29'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:31.050685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf9f78f08444220bd4ee1ffd0b4db29'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:31.060947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 993/1964] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:30 2022] DELETE /identity/v3/users/7bf9f78f08444220bd4ee1ffd0b4db29 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:31.066335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300892f3-e045-4694-8983-16df9b075e86 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:31.066630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300892f3-e045-4694-8983-16df9b075e86 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:31.067178 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300892f3-e045-4694-8983-16df9b075e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:31.087127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-300892f3-e045-4694-8983-16df9b075e86 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:31.088353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-300892f3-e045-4694-8983-16df9b075e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:31.132261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:31.163626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 972/1965] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:31.174560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:31.176070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:31.247796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c24e-fa23-4826-8e60-aa0f4e7d07a2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:31.252459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:31.253291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:31.253420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:31.253662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:31.266806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:31.268356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:31.280306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 994/1966] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:31.620165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300892f3-e045-4694-8983-16df9b075e86 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:31.622099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-300892f3-e045-4694-8983-16df9b075e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:31.721557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300892f3-e045-4694-8983-16df9b075e86 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:31.726017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:31.726722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:31.727017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:31.727381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:31.729266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:31.803034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:31.805026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:31.818251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 973/1967] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:31.931104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:31.933358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:31.973207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf0aa-94de-444e-a00a-3110a2941877 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:31.985038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:31.985961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:31.986316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:31.986669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] PATH_INFO: `/v3/projects/53a35e5bdf7f4b6bb31cd559a4deab99` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:31.987954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a35e5bdf7f4b6bb31cd559a4deab99)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:31.990965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:32.065614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:32.073473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a35e5bdf7f4b6bb31cd559a4deab99', 'request_id': 'req-116455b7-421b-444f-a95e-285ea2bcf234'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.086515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a35e5bdf7f4b6bb31cd559a4deab99', 'request_id': 'req-116455b7-421b-444f-a95e-285ea2bcf234'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.098277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a35e5bdf7f4b6bb31cd559a4deab99', 'request_id': 'req-116455b7-421b-444f-a95e-285ea2bcf234'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.119737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] The token cache is being invalidate because project 53a35e5bdf7f4b6bb31cd559a4deab99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:32.120130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.121043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.126804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 995/1968] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:31 2022] DELETE /identity/v3/projects/53a35e5bdf7f4b6bb31cd559a4deab99 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:32.127929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:32.152910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c10895-66e0-4d9c-bc4c-7cced6ef4e1c service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:32.303858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:32.304522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:32.304827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:32.305200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] PATH_INFO: `/v3/users/643505e75e1d40f3b199e96ed49484d9` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:32.307860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643505e75e1d40f3b199e96ed49484d9)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:32.309822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:32.401093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:32.419626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643505e75e1d40f3b199e96ed49484d9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.420655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643505e75e1d40f3b199e96ed49484d9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.453897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643505e75e1d40f3b199e96ed49484d9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.473052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643505e75e1d40f3b199e96ed49484d9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.473831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8412aa3f-dc0d-4876-ae24-f23a4c675497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643505e75e1d40f3b199e96ed49484d9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:32.509819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 974/1969] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:32 2022] DELETE /identity/v3/users/643505e75e1d40f3b199e96ed49484d9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:32.994165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.001402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:33.028638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116455b7-421b-444f-a95e-285ea2bcf234 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.116762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.117571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.117868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.118149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] PATH_INFO: `/v3/projects/acc8a6b1ba4d4993b7d8350058a928b4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.119684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc8a6b1ba4d4993b7d8350058a928b4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.121808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:33.182485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc8a6b1ba4d4993b7d8350058a928b4', 'request_id': 'req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:33.193098 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc8a6b1ba4d4993b7d8350058a928b4', 'request_id': 'req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:33.200504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc8a6b1ba4d4993b7d8350058a928b4', 'request_id': 'req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:33.214013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] The token cache is being invalidate because project acc8a6b1ba4d4993b7d8350058a928b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:33.214455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:33.215127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-820c54ae-7c4b-4a8e-bb81-2ca7226bdeab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:33.216917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 996/1970] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:32 2022] DELETE /identity/v3/projects/acc8a6b1ba4d4993b7d8350058a928b4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:33.221802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6803a2-7093-4765-b2f9-4f0a39d2698b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.222287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6803a2-7093-4765-b2f9-4f0a39d2698b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.222473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6803a2-7093-4765-b2f9-4f0a39d2698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.253576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2b6803a2-7093-4765-b2f9-4f0a39d2698b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:33.255121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2b6803a2-7093-4765-b2f9-4f0a39d2698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:33.328263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 975/1971] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:33.336199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25e13-a926-4348-85d1-558c6b530ddd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.336497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25e13-a926-4348-85d1-558c6b530ddd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.336805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25e13-a926-4348-85d1-558c6b530ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.358039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7ec25e13-a926-4348-85d1-558c6b530ddd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:33.359822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7ec25e13-a926-4348-85d1-558c6b530ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:33.372658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 997/1972] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:33.373827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6803a2-7093-4765-b2f9-4f0a39d2698b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.386563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6803a2-7093-4765-b2f9-4f0a39d2698b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.399691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.400479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.400888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.401266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.402069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:33.402444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.404153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:33.421721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 976/1973] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:27:33.423002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec25e13-a926-4348-85d1-558c6b530ddd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.433205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec25e13-a926-4348-85d1-558c6b530ddd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.439027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.439731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.440079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.440423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.441175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.443788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:33.479502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 998/1974] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:33 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:33.483506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.495295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88f0b-63a9-4905-ab15-79437e8d33d5 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.578100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.578744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.578979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.579262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.584608 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.586623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:33.612200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 977/1975] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:33 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:33.615933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.627237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd4346-1e35-4529-8ef4-5a4a52487f97 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.632292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.633116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.633418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.633691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.634376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.636043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:33.648020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 999/1976] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:33.652840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.663556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78205de-a570-4d05-bafa-9bc863b6771f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.668616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83a4f-51e5-42f6-9e83-f879d1ccd17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.668954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83a4f-51e5-42f6-9e83-f879d1ccd17f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.669255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83a4f-51e5-42f6-9e83-f879d1ccd17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.669519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83a4f-51e5-42f6-9e83-f879d1ccd17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.670252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83a4f-51e5-42f6-9e83-f879d1ccd17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.672435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83a4f-51e5-42f6-9e83-f879d1ccd17f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:33.691500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 978/1977] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:33.693192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.706566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f247e5-8315-4768-a6ca-239f63d257d8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.713424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.714128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.714631 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.714914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] PATH_INFO: `/v3/projects/bd98617d6b3c43129b3bd7d5b3556b33/users/baa5328e8e1a4ed8b0c1468015aebb48/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.748965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd98617d6b3c43129b3bd7d5b3556b33, user_id=baa5328e8e1a4ed8b0c1468015aebb48, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.752005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:33.763944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:33.770139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1000/1978] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:33 2022] PUT /identity/v3/projects/bd98617d6b3c43129b3bd7d5b3556b33/users/baa5328e8e1a4ed8b0c1468015aebb48/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:33.782985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec32ab-611f-423d-a78c-623d207b94e7 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.783286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec32ab-611f-423d-a78c-623d207b94e7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.783570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec32ab-611f-423d-a78c-623d207b94e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.821719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-45ec32ab-611f-423d-a78c-623d207b94e7 None None] MFA Rules not processed for user `baa5328e8e1a4ed8b0c1468015aebb48`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:33.823470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-45ec32ab-611f-423d-a78c-623d207b94e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:33.831218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:33.894332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 979/1979] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:33.898908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:33.965104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a92db-2c2a-4a5d-8960-717f0d5903e6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:33.977164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:33.977808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:33.978096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:33.978376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:33.979065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:33.981136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.012291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1001/1980] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:33 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:34.016350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec32ab-611f-423d-a78c-623d207b94e7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.027742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec32ab-611f-423d-a78c-623d207b94e7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.095387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.096039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.096328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.096644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.097322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.099355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.136597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 980/1981] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:34 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:34.137838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.150001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d898fd-ed77-48d9-a8e8-eb72f0f5ef88 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.157078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.157710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.158001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.158283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.158978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.160603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.176582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1002/1982] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:34.178722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.189429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee784f4f-b346-46c2-94e1-ec9b9bfc0968 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.194285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.195003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.195287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.195565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] PATH_INFO: `/v3/projects/ded4fd6a7cc943ef880ccf00353c30ff/users/2ab938c139ec431abcbe1625d4baf371/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.214173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded4fd6a7cc943ef880ccf00353c30ff, user_id=2ab938c139ec431abcbe1625d4baf371, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.217754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.231882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 981/1983] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:34 2022] PUT /identity/v3/projects/ded4fd6a7cc943ef880ccf00353c30ff/users/2ab938c139ec431abcbe1625d4baf371/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:34.236087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.246692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718745c5-65f0-4bfb-bfeb-319d1392256e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.315701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106cd62-359e-4e85-80ab-c26a41f91f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.316570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106cd62-359e-4e85-80ab-c26a41f91f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.316948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106cd62-359e-4e85-80ab-c26a41f91f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.317235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106cd62-359e-4e85-80ab-c26a41f91f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.317931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106cd62-359e-4e85-80ab-c26a41f91f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.319510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106cd62-359e-4e85-80ab-c26a41f91f23 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.330381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1003/1984] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:34.334818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.346072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab919f0a-9534-4d20-b97d-4a7096833cde admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.350836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.351510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.351789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.352063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] PATH_INFO: `/v3/projects/ded4fd6a7cc943ef880ccf00353c30ff/users/2ab938c139ec431abcbe1625d4baf371/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.354071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded4fd6a7cc943ef880ccf00353c30ff, user_id=2ab938c139ec431abcbe1625d4baf371, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.356855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.367877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 982/1985] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:34 2022] PUT /identity/v3/projects/ded4fd6a7cc943ef880ccf00353c30ff/users/2ab938c139ec431abcbe1625d4baf371/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:34.379046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94ab6e-0939-4ca3-8a85-c62a2ffc8878 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.379323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94ab6e-0939-4ca3-8a85-c62a2ffc8878 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.379604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94ab6e-0939-4ca3-8a85-c62a2ffc8878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.407901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dd94ab6e-0939-4ca3-8a85-c62a2ffc8878 None None] MFA Rules not processed for user `2ab938c139ec431abcbe1625d4baf371`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:34.409219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dd94ab6e-0939-4ca3-8a85-c62a2ffc8878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:34.471596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:34.484938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1004/1986] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:34.497529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.499470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:34.562480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7711f1-d52a-4758-b7f8-0c9239a8c7a3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.566429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a2283-8c9b-4cd4-9891-fa4f405494fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.567051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2283-8c9b-4cd4-9891-fa4f405494fa service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.567332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2283-8c9b-4cd4-9891-fa4f405494fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.567607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2283-8c9b-4cd4-9891-fa4f405494fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.577243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a2283-8c9b-4cd4-9891-fa4f405494fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.579338 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a2283-8c9b-4cd4-9891-fa4f405494fa service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.590032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 983/1987] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:34.618034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94ab6e-0939-4ca3-8a85-c62a2ffc8878 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.684866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94ab6e-0939-4ca3-8a85-c62a2ffc8878 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.690048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.690706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.690992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.691274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] PATH_INFO: `/v3/users/baa5328e8e1a4ed8b0c1468015aebb48` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.692462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa5328e8e1a4ed8b0c1468015aebb48)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.694414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:34.752356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa5328e8e1a4ed8b0c1468015aebb48'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:34.759623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa5328e8e1a4ed8b0c1468015aebb48'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:34.775513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa5328e8e1a4ed8b0c1468015aebb48'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:34.785327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa5328e8e1a4ed8b0c1468015aebb48'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:34.786147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa5328e8e1a4ed8b0c1468015aebb48'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:34.796095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1005/1988] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:34 2022] DELETE /identity/v3/users/baa5328e8e1a4ed8b0c1468015aebb48 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:34.802921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.803216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.803527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.811058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:34.826679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:34.828620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:34.902526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 984/1989] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:34.913531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:34.915070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:34.976031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f56e8-3972-4644-978c-8ff90ec29b16 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:34.979968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:34.980645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:34.980937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:34.981216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:34.991748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:34.993364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:35.004344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1006/1990] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:35.289801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:35.291261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:35.305380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca24558-d1ae-4aa9-9afa-f7ba0ea49eaf None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:35.311004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:35.312372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:35.312926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:35.313209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] PATH_INFO: `/v3/projects/bd98617d6b3c43129b3bd7d5b3556b33` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:35.314420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd98617d6b3c43129b3bd7d5b3556b33)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:35.316387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:35.365002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd98617d6b3c43129b3bd7d5b3556b33', 'request_id': 'req-ca3eee5d-1993-4693-90af-b69fa7c033b2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.371366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd98617d6b3c43129b3bd7d5b3556b33', 'request_id': 'req-ca3eee5d-1993-4693-90af-b69fa7c033b2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.379504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd98617d6b3c43129b3bd7d5b3556b33', 'request_id': 'req-ca3eee5d-1993-4693-90af-b69fa7c033b2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.391120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] The token cache is being invalidate because project bd98617d6b3c43129b3bd7d5b3556b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:35.391415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.392069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.393649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 985/1991] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:35 2022] DELETE /identity/v3/projects/bd98617d6b3c43129b3bd7d5b3556b33 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:35.397730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:35.407853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecec78-8232-4db4-a51e-53b0454ff42e service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:35.420818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:35.472742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:35.473397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:35.473681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:35.473962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] PATH_INFO: `/v3/users/2ab938c139ec431abcbe1625d4baf371` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:35.475136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab938c139ec431abcbe1625d4baf371)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:35.477117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:35.524367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab938c139ec431abcbe1625d4baf371'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.531956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab938c139ec431abcbe1625d4baf371'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.543751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab938c139ec431abcbe1625d4baf371'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.551105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab938c139ec431abcbe1625d4baf371'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.551809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdbc46d8-723b-451f-8a9b-cb1d43c1439f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab938c139ec431abcbe1625d4baf371'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:35.557790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1007/1992] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:35 2022] DELETE /identity/v3/users/2ab938c139ec431abcbe1625d4baf371 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:35.839755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:35.841412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:35.856192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eee5d-1993-4693-90af-b69fa7c033b2 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:35.988389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:35.989419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:35.989966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:35.991479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] PATH_INFO: `/v3/projects/ded4fd6a7cc943ef880ccf00353c30ff` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:35.993557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded4fd6a7cc943ef880ccf00353c30ff)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.005930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.082331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:36.113792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded4fd6a7cc943ef880ccf00353c30ff', 'request_id': 'req-23d8619b-7826-40ec-b4fe-672bc3733581'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:36.123288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded4fd6a7cc943ef880ccf00353c30ff', 'request_id': 'req-23d8619b-7826-40ec-b4fe-672bc3733581'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:36.135496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded4fd6a7cc943ef880ccf00353c30ff', 'request_id': 'req-23d8619b-7826-40ec-b4fe-672bc3733581'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:36.151150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] The token cache is being invalidate because project ded4fd6a7cc943ef880ccf00353c30ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:36.151586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:36.152928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-23d8619b-7826-40ec-b4fe-672bc3733581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:36.154719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 986/1993] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:35 2022] DELETE /identity/v3/projects/ded4fd6a7cc943ef880ccf00353c30ff => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:36.160414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1c83-8201-47b3-83c7-7bde217ba718 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.160761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1c83-8201-47b3-83c7-7bde217ba718 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.161054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1c83-8201-47b3-83c7-7bde217ba718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.189071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c54c1c83-8201-47b3-83c7-7bde217ba718 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:36.190324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c54c1c83-8201-47b3-83c7-7bde217ba718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:36.265012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1008/1994] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:36.277918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7885db-b6e6-43c1-b7a1-f90181f7c68a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.278430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7885db-b6e6-43c1-b7a1-f90181f7c68a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.278875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7885db-b6e6-43c1-b7a1-f90181f7c68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.307869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-bc7885db-b6e6-43c1-b7a1-f90181f7c68a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:36.309306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bc7885db-b6e6-43c1-b7a1-f90181f7c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:36.319918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 987/1995] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:36.324288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1c83-8201-47b3-83c7-7bde217ba718 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.337326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1c83-8201-47b3-83c7-7bde217ba718 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:36.343443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:36.344769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.345061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.345433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.346050 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:36.346361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.348072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.362281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1009/1996] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:27:36.367070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7885db-b6e6-43c1-b7a1-f90181f7c68a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.383212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7885db-b6e6-43c1-b7a1-f90181f7c68a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:36.389933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:36.391318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.391702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.392347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.393352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.395882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.432337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 988/1997] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:36 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:36.435706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.453923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6c008-a4c7-4044-954a-c6253f4cc91b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:36.487461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:36.569997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:36.570394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.570680 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.570964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.571649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.574090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.600664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1010/1998] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:36 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:36.605750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.618647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd4440-f97e-41ed-9508-20e257cdbcd9 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:36.624065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:36.624791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.625063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.625342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.626041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.627627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.640483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 989/1999] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:36.644826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.657306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9feb2-7d36-4e93-a866-995a44ff437c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:36.662552 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd712f3-596a-4a47-b918-25c26b34bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:36.663183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd712f3-596a-4a47-b918-25c26b34bce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.663473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd712f3-596a-4a47-b918-25c26b34bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.663754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd712f3-596a-4a47-b918-25c26b34bce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.664452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd712f3-596a-4a47-b918-25c26b34bce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.666096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd712f3-596a-4a47-b918-25c26b34bce0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.677199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1011/2000] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:36.681577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.691937 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609e40f-0088-49c7-a865-dc32f491cf74 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:36.697071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:36.697745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.698023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.698299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] PATH_INFO: `/v3/projects/3b532b4974e64396a6eaaa7688510a0b/users/fcc4c08106ce41089b3748f439390db8/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.714640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b532b4974e64396a6eaaa7688510a0b, user_id=fcc4c08106ce41089b3748f439390db8, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.720376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.735684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 990/2001] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:36 2022] PUT /identity/v3/projects/3b532b4974e64396a6eaaa7688510a0b/users/fcc4c08106ce41089b3748f439390db8/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:36.748677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a30f31-74d4-40b3-ace8-552f4e1b8fa4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.748946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a30f31-74d4-40b3-ace8-552f4e1b8fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.749830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a30f31-74d4-40b3-ace8-552f4e1b8fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.781088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-52a30f31-74d4-40b3-ace8-552f4e1b8fa4 None None] MFA Rules not processed for user `fcc4c08106ce41089b3748f439390db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:36.782335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-52a30f31-74d4-40b3-ace8-552f4e1b8fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:36.853684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1012/2002] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:36.858073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.931361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044588f5-8e15-4ef5-b24e-64c8dbb774d9 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:36.936356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:36.937080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:36.937811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:36.937811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:36.938294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:36.940182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:36.975423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 991/2003] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:36 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:36.979438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a30f31-74d4-40b3-ace8-552f4e1b8fa4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:36.992934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a30f31-74d4-40b3-ace8-552f4e1b8fa4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:37.060739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:37.061377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.061664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.062504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.063465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:37.065740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:37.108197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1013/2004] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:36 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:37.110628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:37.124156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f47d23-1d14-46e3-8c45-f13e2c291c26 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:37.141339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:37.142066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.142443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.142794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.143776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:37.145768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:37.164656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 992/2005] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:37.167190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:37.182544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d00f20-bbfa-4287-a35f-444c97bb5b00 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:37.204879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:37.205899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.206389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.206815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] PATH_INFO: `/v3/projects/f2d3c0e624d14883a9f2307e8afde615/users/c30e10df5659459eb2f56a33d8a2e123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.227477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d3c0e624d14883a9f2307e8afde615, user_id=c30e10df5659459eb2f56a33d8a2e123, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:37.230465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:37.235678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:37.257356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1014/2006] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:37 2022] PUT /identity/v3/projects/f2d3c0e624d14883a9f2307e8afde615/users/c30e10df5659459eb2f56a33d8a2e123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:37.261309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:37.282239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875d597-66b6-4c18-b473-823160e510e5 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:37.329109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:37.406574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db645bf-a756-4f36-abb5-b5d7b62733c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:37.407355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db645bf-a756-4f36-abb5-b5d7b62733c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.407643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db645bf-a756-4f36-abb5-b5d7b62733c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.407925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db645bf-a756-4f36-abb5-b5d7b62733c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.408700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db645bf-a756-4f36-abb5-b5d7b62733c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:37.410335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db645bf-a756-4f36-abb5-b5d7b62733c2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:37.437728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 993/2007] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:37.437853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:37.445072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418d010-3bf4-4ce1-aa02-b7db4f980398 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:37.452185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:37.453997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.454386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.454770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] PATH_INFO: `/v3/projects/f2d3c0e624d14883a9f2307e8afde615/users/c30e10df5659459eb2f56a33d8a2e123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.459925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d3c0e624d14883a9f2307e8afde615, user_id=c30e10df5659459eb2f56a33d8a2e123, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:37.462742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:37.476405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1015/2008] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:37 2022] PUT /identity/v3/projects/f2d3c0e624d14883a9f2307e8afde615/users/c30e10df5659459eb2f56a33d8a2e123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:37.487006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.487302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.487585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.530454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] MFA Rules not processed for user `c30e10df5659459eb2f56a33d8a2e123`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:37.532980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:37.622919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 994/2009] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:37.646822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:37.649028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:37.749193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a506cd4-8713-4e5e-9ea9-ffd7f010a11b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:37.754692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dbcae-60ab-418c-8a0d-1631395decb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:37.755489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dbcae-60ab-418c-8a0d-1631395decb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.755778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dbcae-60ab-418c-8a0d-1631395decb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.756094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dbcae-60ab-418c-8a0d-1631395decb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.766881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dbcae-60ab-418c-8a0d-1631395decb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:37.768325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dbcae-60ab-418c-8a0d-1631395decb9 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:37.778241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1016/2010] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:37.832367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:37.834043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:37.845892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09090-443f-4a27-86b4-81fae137dd9b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:37.850433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673471e3-ba78-4db7-aad0-3c8d1fa92d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:37.851051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673471e3-ba78-4db7-aad0-3c8d1fa92d87 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:37.851332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673471e3-ba78-4db7-aad0-3c8d1fa92d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:37.851606 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673471e3-ba78-4db7-aad0-3c8d1fa92d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:37.919560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673471e3-ba78-4db7-aad0-3c8d1fa92d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:37.921067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673471e3-ba78-4db7-aad0-3c8d1fa92d87 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:37.932036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 995/2011] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:38.338457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031a25-875b-488a-9901-9af49d003745 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:38.339026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031a25-875b-488a-9901-9af49d003745 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:38.339448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031a25-875b-488a-9901-9af49d003745 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:38.341122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1017/2012] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:27:38.352274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494810dc-c9fa-42ce-82fd-97bc97507805 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:38.352831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494810dc-c9fa-42ce-82fd-97bc97507805 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:38.352943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494810dc-c9fa-42ce-82fd-97bc97507805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:38.373686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-494810dc-c9fa-42ce-82fd-97bc97507805 None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:38.376657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-494810dc-c9fa-42ce-82fd-97bc97507805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:38.425145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:38.449764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 996/2013] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:27:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:38.467362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031a25-875b-488a-9901-9af49d003745 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:38.471307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-70031a25-875b-488a-9901-9af49d003745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:38.491583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031a25-875b-488a-9901-9af49d003745 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:38.497514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2f6bd-5dd6-4136-bf90-eda19e35478f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:38.498215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f6bd-5dd6-4136-bf90-eda19e35478f service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:38.498484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f6bd-5dd6-4136-bf90-eda19e35478f service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:38.498765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f6bd-5dd6-4136-bf90-eda19e35478f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:38.509804 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2f6bd-5dd6-4136-bf90-eda19e35478f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:38.511870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2f6bd-5dd6-4136-bf90-eda19e35478f service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:38.523683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1018/2014] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:38.745091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494810dc-c9fa-42ce-82fd-97bc97507805 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:38.755048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494810dc-c9fa-42ce-82fd-97bc97507805 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:38.758779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb87725-e7d8-4885-b502-6b1de67e5cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:38.759591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb87725-e7d8-4885-b502-6b1de67e5cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:38.759841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb87725-e7d8-4885-b502-6b1de67e5cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:38.760204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb87725-e7d8-4885-b502-6b1de67e5cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:38.762112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6cb87725-e7d8-4885-b502-6b1de67e5cc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:38.833631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb87725-e7d8-4885-b502-6b1de67e5cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:38.835111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb87725-e7d8-4885-b502-6b1de67e5cc5 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:38.844483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 997/2015] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:39.115145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9511-7ed8-457b-a647-cb5786269c40 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:39.115460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9511-7ed8-457b-a647-cb5786269c40 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:39.115721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa9511-7ed8-457b-a647-cb5786269c40 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:39.117285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1019/2016] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:27:39.151523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7fdc7-5968-4487-81e2-2231ce6bce01 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:39.152018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7fdc7-5968-4487-81e2-2231ce6bce01 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:39.152434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7fdc7-5968-4487-81e2-2231ce6bce01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:39.176112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7fb7fdc7-5968-4487-81e2-2231ce6bce01 None None] MFA Rules not processed for user `b4986a5d3e6c4adf832c1b1ec8dd768a`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:39.177607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7fb7fdc7-5968-4487-81e2-2231ce6bce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:39.190222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 998/2017] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:27:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:39.193255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9511-7ed8-457b-a647-cb5786269c40 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:39.195185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4faa9511-7ed8-457b-a647-cb5786269c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:39.204177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7fdc7-5968-4487-81e2-2231ce6bce01 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:39.217956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7fdc7-5968-4487-81e2-2231ce6bce01 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:39.223532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:39.224461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:39.225110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:39.225535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:39.235502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:39.237432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:39.252116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 999/2018] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:39.306559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa9511-7ed8-457b-a647-cb5786269c40 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:39.313478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:39.314096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:39.314379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:39.314750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] PATH_INFO: `/v3/users/fc1e741febab41f0a8cefedf60752c22` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:39.316235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1e741febab41f0a8cefedf60752c22)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:39.318277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:39.384325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1e741febab41f0a8cefedf60752c22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:39.393898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1e741febab41f0a8cefedf60752c22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:39.409390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1e741febab41f0a8cefedf60752c22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:39.414213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:39.426157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1e741febab41f0a8cefedf60752c22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:39.427749 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1e741febab41f0a8cefedf60752c22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:39.443781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1020/2019] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:39 2022] DELETE /identity/v3/users/fc1e741febab41f0a8cefedf60752c22 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:39.886284 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:39.900766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539e795-ba0b-4a47-8eee-de9397d22312 service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:39.972985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:39.975716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:39.976805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:39.977832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] PATH_INFO: `/v3/projects/865414b5436742a39fd016dddd34d556` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:39.980472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=865414b5436742a39fd016dddd34d556)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:39.984304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:40.071431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865414b5436742a39fd016dddd34d556', 'request_id': 'req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.079838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865414b5436742a39fd016dddd34d556', 'request_id': 'req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.095670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865414b5436742a39fd016dddd34d556', 'request_id': 'req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.098974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:40.117673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] The token cache is being invalidate because project 865414b5436742a39fd016dddd34d556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:40.118324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.119292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.121948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1000/2020] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:39 2022] DELETE /identity/v3/projects/865414b5436742a39fd016dddd34d556 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:40.126829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:40.128570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:40.155187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4439-0e42-43e8-bf18-1a460c6ff0d9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:40.268763 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:40.269871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:40.270547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:40.271021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] PATH_INFO: `/v3/users/b2c658686da74fafad632e61704bdc22` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:40.285655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c658686da74fafad632e61704bdc22)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:40.293024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:40.374031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c658686da74fafad632e61704bdc22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.382925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c658686da74fafad632e61704bdc22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.399511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c658686da74fafad632e61704bdc22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.412877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c658686da74fafad632e61704bdc22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.416997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-41b0a3b8-e6e2-4297-a992-99e12da7c0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c658686da74fafad632e61704bdc22'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.425786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1021/2021] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:40 2022] DELETE /identity/v3/users/b2c658686da74fafad632e61704bdc22 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:40.614682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:40.616579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:40.631626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3ff8c-b46f-426d-b120-b99a7904ac6e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:40.700520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:40.700867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:40.701298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:40.701646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] PATH_INFO: `/v3/projects/e7093a96fb8d43f9afd0e198be28a713` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:40.703701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7093a96fb8d43f9afd0e198be28a713)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:40.706637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:40.774999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7093a96fb8d43f9afd0e198be28a713', 'request_id': 'req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.782001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7093a96fb8d43f9afd0e198be28a713', 'request_id': 'req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.790271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7093a96fb8d43f9afd0e198be28a713', 'request_id': 'req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.806600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] The token cache is being invalidate because project e7093a96fb8d43f9afd0e198be28a713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:40.806891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.807549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4e70b7f1-cfa5-4279-9dbb-43d3f3e14873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:40.809188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1001/2022] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:40 2022] DELETE /identity/v3/projects/e7093a96fb8d43f9afd0e198be28a713 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:40.815985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcbfc7-1563-4863-b5fe-429f79dfe7d7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:40.816352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcbfc7-1563-4863-b5fe-429f79dfe7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:40.816895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcbfc7-1563-4863-b5fe-429f79dfe7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:40.838144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7fdcbfc7-1563-4863-b5fe-429f79dfe7d7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:40.839541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7fdcbfc7-1563-4863-b5fe-429f79dfe7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:40.910044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1022/2023] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:40.917545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08974a27-683c-4817-9325-83b274709513 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:40.918038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08974a27-683c-4817-9325-83b274709513 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:40.918130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08974a27-683c-4817-9325-83b274709513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:40.938283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-08974a27-683c-4817-9325-83b274709513 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:40.939489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-08974a27-683c-4817-9325-83b274709513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:40.947592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1002/2024] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:40.951659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcbfc7-1563-4863-b5fe-429f79dfe7d7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:40.962989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcbfc7-1563-4863-b5fe-429f79dfe7d7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:40.967939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:40.969409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:40.969698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:40.969981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:40.970676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:40.971020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:40.972630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:40.980078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:40.988822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1023/2025] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:40.991821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08974a27-683c-4817-9325-83b274709513 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.002550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08974a27-683c-4817-9325-83b274709513 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.007601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.008223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.008514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.008825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.009524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.011805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.053547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1003/2026] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:40 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:41.054955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.065701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740006a-88f5-4992-8926-dc82a2e2d7f6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.135580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.136319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.136617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.136874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.137592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.140761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.177259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1024/2027] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:41 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:41.180896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.207575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d7106-2456-4d5f-9f3c-d7811e30fe24 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.221771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.222463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.222921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.223978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.225801 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.227923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.243023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1004/2028] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:41.244143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.263076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5a67c-a2da-479f-9402-1e8f6e280c72 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.269880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.270678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.270915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.271153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.282889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.284895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:41.288875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.290462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.300139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6776ad-1d1a-4fc7-b0c1-673f73b78b3b admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.308362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1025/2029] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:41.308670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.309396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.309687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.309974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] PATH_INFO: `/v3/users/ee26289aef2d42e5bb6412df96cbfb72` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.311480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee26289aef2d42e5bb6412df96cbfb72)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.312846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.314144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.325849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ee9a1-b1f0-48bb-ac28-33875eccc6e0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.331472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88908f-13b7-49ff-8683-69c8f3089396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.333043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88908f-13b7-49ff-8683-69c8f3089396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.333128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88908f-13b7-49ff-8683-69c8f3089396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.333460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88908f-13b7-49ff-8683-69c8f3089396 admin admin] PATH_INFO: `/v3/projects/c241dc0f043e46c8a01503a1c343abf9/users/55bb264bd9834f5fa61b58362ec48d31/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.350148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88908f-13b7-49ff-8683-69c8f3089396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c241dc0f043e46c8a01503a1c343abf9, user_id=55bb264bd9834f5fa61b58362ec48d31, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.353474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88908f-13b7-49ff-8683-69c8f3089396 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.362305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:41.372060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1026/2030] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:41 2022] PUT /identity/v3/projects/c241dc0f043e46c8a01503a1c343abf9/users/55bb264bd9834f5fa61b58362ec48d31/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:41.373783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee26289aef2d42e5bb6412df96cbfb72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.375025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee26289aef2d42e5bb6412df96cbfb72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.377469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.377794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.378076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.388630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee26289aef2d42e5bb6412df96cbfb72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.396942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee26289aef2d42e5bb6412df96cbfb72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.397667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d3b6a85e-f0a3-441c-a239-19684871980b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee26289aef2d42e5bb6412df96cbfb72'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.399284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:41.400527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:41.418521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1005/2031] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:41 2022] DELETE /identity/v3/users/ee26289aef2d42e5bb6412df96cbfb72 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:41.423415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.423731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.424017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.444269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:41.445554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:41.476084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1027/2032] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:41.488292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.489681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:41.521962 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1006/2033] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:41.533048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.534534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:41.570274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e209bb6-7c81-403b-bcb8-20fea5f7cc60 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.579077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:41.612166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a5f5-941e-43e8-90ae-5d0694d6311d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.618291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.619414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.620066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.620352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.637371 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.639293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.642328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9c97c-02d9-49b0-8247-4c67236ccff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.643001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c97c-02d9-49b0-8247-4c67236ccff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.643281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c97c-02d9-49b0-8247-4c67236ccff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.643561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c97c-02d9-49b0-8247-4c67236ccff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.658071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9c97c-02d9-49b0-8247-4c67236ccff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.659750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9c97c-02d9-49b0-8247-4c67236ccff8 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.662039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1007/2034] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:41.675214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1028/2035] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:41.763462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:41.765665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:41.781401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4225c37-ed0c-4a39-994c-5b24d6e1f05f service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:41.789443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:41.790160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.790467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.790744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] PATH_INFO: `/v3/users/fcc4c08106ce41089b3748f439390db8` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.791951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc4c08106ce41089b3748f439390db8)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:41.793932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:41.872513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc4c08106ce41089b3748f439390db8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.873660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc4c08106ce41089b3748f439390db8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.879982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:41.892686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc4c08106ce41089b3748f439390db8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.902918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc4c08106ce41089b3748f439390db8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.903805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc4c08106ce41089b3748f439390db8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:41.921703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1008/2036] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:41 2022] DELETE /identity/v3/users/fcc4c08106ce41089b3748f439390db8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:41.926219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:41.926646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:41.926989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:41.946423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:41.947981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:42.015927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1029/2037] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:42.036167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:42.038055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:42.129350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccd261-728b-4482-8019-52be2ea500c5 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:42.133589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:42.135041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:42.135337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:42.135781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:42.151020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:42.153022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:42.172231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:42.173725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:42.176627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1009/2038] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:42.208350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4e26b-084a-4b82-8aba-853961da0d50 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:42.215592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:42.216380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:42.217642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:42.218675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] PATH_INFO: `/v3/projects/7b329e7029d64789acfb42526d6741f8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:42.220744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b329e7029d64789acfb42526d6741f8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:42.223677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:42.323302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b329e7029d64789acfb42526d6741f8', 'request_id': 'req-fd8d0d69-beea-42f0-b3e4-87e25438897b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.339557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b329e7029d64789acfb42526d6741f8', 'request_id': 'req-fd8d0d69-beea-42f0-b3e4-87e25438897b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.360955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b329e7029d64789acfb42526d6741f8', 'request_id': 'req-fd8d0d69-beea-42f0-b3e4-87e25438897b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.398714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] The token cache is being invalidate because project 7b329e7029d64789acfb42526d6741f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:42.399001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.399851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.407241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:42.412493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1030/2039] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:42 2022] DELETE /identity/v3/projects/7b329e7029d64789acfb42526d6741f8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:42.423893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bf9dc-7f7c-48b3-bf21-fb02f9bf6a42 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:42.595637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:42.596369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:42.596879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:42.597157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] PATH_INFO: `/v3/users/c1af69dcee0f4ed7803d400a40fb8ff3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:42.599609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1af69dcee0f4ed7803d400a40fb8ff3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:42.604099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:42.687315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1af69dcee0f4ed7803d400a40fb8ff3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.688003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1af69dcee0f4ed7803d400a40fb8ff3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.706233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1af69dcee0f4ed7803d400a40fb8ff3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.722556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1af69dcee0f4ed7803d400a40fb8ff3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.723687 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-80c6e5af-13e3-4cbc-967f-8b684bc77c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1af69dcee0f4ed7803d400a40fb8ff3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:42.725901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:42.745495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1010/2040] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:42 2022] DELETE /identity/v3/users/c1af69dcee0f4ed7803d400a40fb8ff3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:42.771498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:42.773192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:42.813577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d0d69-beea-42f0-b3e4-87e25438897b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:42.883809 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:42.891686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:42.891999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:42.892382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:42.927211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] MFA Rules not processed for user `55bb264bd9834f5fa61b58362ec48d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:42.929472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:42.958026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:42.958727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:42.959012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:42.959291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] PATH_INFO: `/v3/projects/3b532b4974e64396a6eaaa7688510a0b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:42.962464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b532b4974e64396a6eaaa7688510a0b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:42.976701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:43.041561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b532b4974e64396a6eaaa7688510a0b', 'request_id': 'req-4df95c03-492b-4f30-8e5d-ffb57aa51f67'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.064828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b532b4974e64396a6eaaa7688510a0b', 'request_id': 'req-4df95c03-492b-4f30-8e5d-ffb57aa51f67'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.076360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b532b4974e64396a6eaaa7688510a0b', 'request_id': 'req-4df95c03-492b-4f30-8e5d-ffb57aa51f67'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.088712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1011/2041] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:43.090007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:43.092644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:43.105776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] The token cache is being invalidate because project 3b532b4974e64396a6eaaa7688510a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:43.106276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.107129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.109949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1031/2042] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:42 2022] DELETE /identity/v3/projects/3b532b4974e64396a6eaaa7688510a0b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:43.113996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:43.115696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:43.166152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a0f4-ac24-4e40-9a47-68a5ef28e336 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:43.215223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df95c03-492b-4f30-8e5d-ffb57aa51f67 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:43.221107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:43.221873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:43.222232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:43.222587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] PATH_INFO: `/v3/users/c30e10df5659459eb2f56a33d8a2e123` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:43.233970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c30e10df5659459eb2f56a33d8a2e123)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:43.236853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:43.255873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:43.256512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:43.256818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:43.257095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:43.262049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:43.315901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c30e10df5659459eb2f56a33d8a2e123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.325249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c30e10df5659459eb2f56a33d8a2e123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.335371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:43.340826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:43.343358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c30e10df5659459eb2f56a33d8a2e123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.357420 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:43.359849 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:43.367591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c30e10df5659459eb2f56a33d8a2e123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.368967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c30e10df5659459eb2f56a33d8a2e123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.372736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1012/2043] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:43.377759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1032/2044] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:43 2022] DELETE /identity/v3/users/c30e10df5659459eb2f56a33d8a2e123 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:43.548323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:43.552814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:43.586864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8120135-82f9-4268-bd69-5d521b050349 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:43.661104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:43.661943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:43.662236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:43.662671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] PATH_INFO: `/v3/projects/218ad380043e4ceb93d19ec31b2e45d0` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:43.664039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=218ad380043e4ceb93d19ec31b2e45d0)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:43.665986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:43.735598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218ad380043e4ceb93d19ec31b2e45d0', 'request_id': 'req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.739978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:43.742045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:43.759318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218ad380043e4ceb93d19ec31b2e45d0', 'request_id': 'req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.773282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218ad380043e4ceb93d19ec31b2e45d0', 'request_id': 'req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.773499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a2ae8-3a1e-4916-ac45-bf491b1e6d52 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:43.791430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] The token cache is being invalidate because project 218ad380043e4ceb93d19ec31b2e45d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:43.791755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.792424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.794749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1013/2045] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:43 2022] DELETE /identity/v3/projects/218ad380043e4ceb93d19ec31b2e45d0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:43.799011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:43.801779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:43.869308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:43.869978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:43.870372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:43.870534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] PATH_INFO: `/v3/projects/f2d3c0e624d14883a9f2307e8afde615` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:43.872167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d3c0e624d14883a9f2307e8afde615)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:43.875168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:43.909161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b5d01-fca0-4cc1-9eff-a9fdeeb42093 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:43.915318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:43.915969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:43.916474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:43.916978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] PATH_INFO: `/v3/users/eae637e08cdc428e8e029a53f128d70b` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:43.919815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae637e08cdc428e8e029a53f128d70b)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:43.922472 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:43.927967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d3c0e624d14883a9f2307e8afde615', 'request_id': 'req-33520e91-e0dc-4c24-b48f-dde00470f2fe'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.940817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d3c0e624d14883a9f2307e8afde615', 'request_id': 'req-33520e91-e0dc-4c24-b48f-dde00470f2fe'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.948950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d3c0e624d14883a9f2307e8afde615', 'request_id': 'req-33520e91-e0dc-4c24-b48f-dde00470f2fe'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.953173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:43.961689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] The token cache is being invalidate because project f2d3c0e624d14883a9f2307e8afde615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:43.962045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.962753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33520e91-e0dc-4c24-b48f-dde00470f2fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.964729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1033/2046] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:43 2022] DELETE /identity/v3/projects/f2d3c0e624d14883a9f2307e8afde615 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:43.970355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1a8a7-2ba5-4c6e-937f-506a0e3cc987 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:43.970722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1a8a7-2ba5-4c6e-937f-506a0e3cc987 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:43.970994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1a8a7-2ba5-4c6e-937f-506a0e3cc987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:43.982839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae637e08cdc428e8e029a53f128d70b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.983609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae637e08cdc428e8e029a53f128d70b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:43.992649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c7e1a8a7-2ba5-4c6e-937f-506a0e3cc987 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:43.993939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c7e1a8a7-2ba5-4c6e-937f-506a0e3cc987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:43.997947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae637e08cdc428e8e029a53f128d70b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.007179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae637e08cdc428e8e029a53f128d70b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.008106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-98060ab4-ec04-4967-ac6d-6dc0d6c84bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae637e08cdc428e8e029a53f128d70b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.037529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1014/2047] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:43 2022] DELETE /identity/v3/users/eae637e08cdc428e8e029a53f128d70b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:44.069186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1034/2048] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:44.077851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e129-9697-4ade-8f03-f04e6389a0f5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.078496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e129-9697-4ade-8f03-f04e6389a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.078796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e129-9697-4ade-8f03-f04e6389a0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.107331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3866e129-9697-4ade-8f03-f04e6389a0f5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:44.109358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3866e129-9697-4ade-8f03-f04e6389a0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:44.184126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1015/2049] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:44.189191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1a8a7-2ba5-4c6e-937f-506a0e3cc987 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.201422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1a8a7-2ba5-4c6e-937f-506a0e3cc987 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.207289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.209161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.209791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.210077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.210803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:44.211102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.212940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.231323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1035/2050] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:44.235691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866e129-9697-4ade-8f03-f04e6389a0f5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.246394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866e129-9697-4ade-8f03-f04e6389a0f5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.252526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.253278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.253628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.254129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.255075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.257111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.300947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1016/2051] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/projects => generated 399 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:44.302456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.308422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:44.314872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8f127-be34-4cad-a46e-73006cf83de4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.373645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.375940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:44.411230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.411880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.412172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.413124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.413927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.415725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.438876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1036/2052] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:44.444098 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.456107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73751cdd-09fa-48b2-a5b6-baf0d2f9750b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.461960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.462654 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.463029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.463380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.464143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.465882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.473913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba7adc-17b3-4db9-bb44-5eb2d082868a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.478185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1037/2053] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:44.479152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.479805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.480086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.480362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] PATH_INFO: `/v3/projects/361775ce832442ae9a70df64707f78bf` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.481820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=361775ce832442ae9a70df64707f78bf)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.482075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.483840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.492681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a875c5-b249-46be-a021-60078fb562ee admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.498132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.498755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.499579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.499853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.500557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.502283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.524917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:44.529644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1038/2054] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:44.535060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.552697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b5e4d-05f3-4beb-b0d0-bfc1c170b476 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.578930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '361775ce832442ae9a70df64707f78bf', 'request_id': 'req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.586529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '361775ce832442ae9a70df64707f78bf', 'request_id': 'req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.597618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '361775ce832442ae9a70df64707f78bf', 'request_id': 'req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.611890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] The token cache is being invalidate because project 361775ce832442ae9a70df64707f78bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:44.612233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.613403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6f3b310-d016-4eb8-b5d5-ef53f76a2a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:44.620579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1017/2055] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:44 2022] DELETE /identity/v3/projects/361775ce832442ae9a70df64707f78bf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:44.622234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd6121-7acb-4d39-a35a-8dc366f2cca5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.622533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd6121-7acb-4d39-a35a-8dc366f2cca5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.622815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd6121-7acb-4d39-a35a-8dc366f2cca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.657860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-18cd6121-7acb-4d39-a35a-8dc366f2cca5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:44.658705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b46f8-4207-4f74-ab5c-21a8ff47e1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.659075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-18cd6121-7acb-4d39-a35a-8dc366f2cca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:44.660027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b46f8-4207-4f74-ab5c-21a8ff47e1de admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.660326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b46f8-4207-4f74-ab5c-21a8ff47e1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.660638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b46f8-4207-4f74-ab5c-21a8ff47e1de admin admin] PATH_INFO: `/v3/projects/a81e9c34f6a9482ca8a9b9965aded674/users/b553ac3f41f24bcca3b6a3ae730f05e9/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.668878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1018/2056] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:44.678431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d60e24-9630-449c-befd-36d843481197 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.678716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d60e24-9630-449c-befd-36d843481197 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.678998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d60e24-9630-449c-befd-36d843481197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.681401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b46f8-4207-4f74-ab5c-21a8ff47e1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a81e9c34f6a9482ca8a9b9965aded674, user_id=b553ac3f41f24bcca3b6a3ae730f05e9, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.684988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b46f8-4207-4f74-ab5c-21a8ff47e1de admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.701410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a2d60e24-9630-449c-befd-36d843481197 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:44.703357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a2d60e24-9630-449c-befd-36d843481197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:44.706783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1039/2057] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:44 2022] PUT /identity/v3/projects/a81e9c34f6a9482ca8a9b9965aded674/users/b553ac3f41f24bcca3b6a3ae730f05e9/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:44.726566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.727297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.727596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.764995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] MFA Rules not processed for user `b553ac3f41f24bcca3b6a3ae730f05e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:44.766320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:44.789899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1019/2058] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:44.794213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d60e24-9630-449c-befd-36d843481197 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.804821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d60e24-9630-449c-befd-36d843481197 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.810340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.810983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.811261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.812114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.813034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:44.813218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.814884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.815371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:44.847452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1020/2059] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:27:44.848405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.854513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1040/2060] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:44.860091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.860385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a09505-0745-4c9b-9ec3-9d40119b5f0b admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.862307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:44.865418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1575-cc66-4b23-a6c1-98b8f00c4141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.866053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1575-cc66-4b23-a6c1-98b8f00c4141 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.866335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1575-cc66-4b23-a6c1-98b8f00c4141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.866611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1575-cc66-4b23-a6c1-98b8f00c4141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.867278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc1575-cc66-4b23-a6c1-98b8f00c4141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.869221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc1575-cc66-4b23-a6c1-98b8f00c4141 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.893539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9022c06-153a-4458-acc8-4da365685f07 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:44.901221 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:44.901898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.902318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.902696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.903383 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:44.905408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:44.910062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1021/2061] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:44.915183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffed13-e695-422a-a9ec-851799ac8d5f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:44.915717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffed13-e695-422a-a9ec-851799ac8d5f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:44.916064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffed13-e695-422a-a9ec-851799ac8d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:44.940615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1041/2062] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/projects => generated 401 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:44.941181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:44.943515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-a7ffed13-e695-422a-a9ec-851799ac8d5f None None] MFA Rules not processed for user `55bb264bd9834f5fa61b58362ec48d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:44.946158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a7ffed13-e695-422a-a9ec-851799ac8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:45.012060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1022/2063] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:45.013059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffed13-e695-422a-a9ec-851799ac8d5f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.025109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce508e5-bdfb-4ea1-b38f-e33af85512c3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.029228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffed13-e695-422a-a9ec-851799ac8d5f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.031917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.032580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.032876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.033161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.034816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.035182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.035441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.035782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.036063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.036854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.037854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.038821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.056935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:45.080638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1023/2064] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:45 2022] POST /identity/v3/users => generated 307 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:45.085120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.093996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1042/2065] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:44 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:45.098124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.099436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bffe4-6359-4060-bfcc-290df7b7cf58 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.115610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.116476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.116899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.117249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.118057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.119773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.126452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd928307-ad0e-478c-9030-b9b9d7af269f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.131910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.132465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.132793 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.133075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.137224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1024/2066] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:45.137370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.137370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.138357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.147081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1043/2067] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:45.150632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b05005-68e1-4ca4-ba28-4ad3dbea5ee6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.151143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.156286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.157256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.157542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.157818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] PATH_INFO: `/v3/projects/6a5a329774f84244a82852e5bf985d1c/users/52ebef836f4a424bb92c325fb0db81da/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.163277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39483067-cbb3-433b-b696-a6ce6a88b391 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.168832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.169427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.169714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.169997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.170687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.173147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.175190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5a329774f84244a82852e5bf985d1c, user_id=52ebef836f4a424bb92c325fb0db81da, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.177980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.193483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1025/2068] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:45 2022] PUT /identity/v3/projects/6a5a329774f84244a82852e5bf985d1c/users/52ebef836f4a424bb92c325fb0db81da/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:45.194794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1044/2069] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:45.197918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.198972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.210320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce7413-02f2-4b4e-a301-5ea70b9a0c33 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.270333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67548b81-cb78-4dd4-ac02-85c5fb6bb13d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.275705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.276399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.276719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.276998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.277729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.279287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.284132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a4d72-0084-4322-b9c2-4789100b7d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.285076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a4d72-0084-4322-b9c2-4789100b7d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.285364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a4d72-0084-4322-b9c2-4789100b7d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.285645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a4d72-0084-4322-b9c2-4789100b7d5f admin admin] PATH_INFO: `/v3/projects/e3c753ea46ec4ef887997fbde7d53b5d/users/1615ebacf82f42daa5e6bbf8e1194030/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.293652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1026/2070] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:45.302655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.309582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a4d72-0084-4322-b9c2-4789100b7d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c753ea46ec4ef887997fbde7d53b5d, user_id=1615ebacf82f42daa5e6bbf8e1194030, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.313240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a4d72-0084-4322-b9c2-4789100b7d5f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.313727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217e84f-4569-4747-b8a8-280ead87d94f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.318680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de7b9e-a8fd-4e0f-ba5f-2c302f257d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.319564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de7b9e-a8fd-4e0f-ba5f-2c302f257d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.320413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de7b9e-a8fd-4e0f-ba5f-2c302f257d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.320413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de7b9e-a8fd-4e0f-ba5f-2c302f257d08 admin admin] PATH_INFO: `/v3/projects/6a5a329774f84244a82852e5bf985d1c/users/52ebef836f4a424bb92c325fb0db81da/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.323004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de7b9e-a8fd-4e0f-ba5f-2c302f257d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5a329774f84244a82852e5bf985d1c, user_id=52ebef836f4a424bb92c325fb0db81da, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.327843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de7b9e-a8fd-4e0f-ba5f-2c302f257d08 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.338014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1045/2071] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:45 2022] PUT /identity/v3/projects/e3c753ea46ec4ef887997fbde7d53b5d/users/1615ebacf82f42daa5e6bbf8e1194030/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:45.341495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac83e69-4a71-4d0d-a473-1c91102d9103 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.341856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac83e69-4a71-4d0d-a473-1c91102d9103 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.342222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac83e69-4a71-4d0d-a473-1c91102d9103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.351913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1027/2072] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:45 2022] PUT /identity/v3/projects/6a5a329774f84244a82852e5bf985d1c/users/52ebef836f4a424bb92c325fb0db81da/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:45.364341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.365123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.365425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.366786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:45.377096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aac83e69-4a71-4d0d-a473-1c91102d9103 None None] MFA Rules not processed for user `1615ebacf82f42daa5e6bbf8e1194030`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:45.379014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aac83e69-4a71-4d0d-a473-1c91102d9103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:45.408692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] MFA Rules not processed for user `52ebef836f4a424bb92c325fb0db81da`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:45.409847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:45.464179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1046/2073] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:45.468954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac83e69-4a71-4d0d-a473-1c91102d9103 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.506311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1028/2074] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:45 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:45.514047 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.515596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:45.562124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac83e69-4a71-4d0d-a473-1c91102d9103 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.570178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.571111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.571581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.572061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.573039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.575383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:45.578051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.612782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c34892-2b3f-403a-85c1-83df72d5ace5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.616997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.617773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.618052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.618430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.620645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1047/2075] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:45 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:45.622013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.631992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.641148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.653170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1029/2076] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:45.732715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d0c7-449a-47c1-a9da-f577238898cf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.745073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.745998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.746495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.746971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.748167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.751284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.753425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.763783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a5cc6-5078-4747-be9a-894d47e16a5e service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.782557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1048/2077] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:45 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:45.786780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.800218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28173c82-b7bc-420a-8612-fe429bdda46a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.806224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.807133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.808101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.808622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.809558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.812747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.826897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1049/2078] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:45.831258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.838336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:45.842806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a63e37-1a6a-4bb0-85d1-58bd508ee39c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:45.850321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.851452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.851928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.852407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] PATH_INFO: `/v3/projects/227915ca681d49028988693c81019dcc/users/f8ac22c8c242451b8fb506feb3a2502f/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.865669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:45.866950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:45.867814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:45.868601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:45.870364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=227915ca681d49028988693c81019dcc, user_id=f8ac22c8c242451b8fb506feb3a2502f, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.873608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.898794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1050/2079] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:45 2022] PUT /identity/v3/projects/227915ca681d49028988693c81019dcc/users/f8ac22c8c242451b8fb506feb3a2502f/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:45.902799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:45.961781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:45.963231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:45.974425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1030/2080] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:46.011939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce388d8a-7d9d-4e66-95c0-3f219eec6a0f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:46.017694 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a442e2-ad7c-4622-bfca-c7ca31df2449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:46.018493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a442e2-ad7c-4622-bfca-c7ca31df2449 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:46.018798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a442e2-ad7c-4622-bfca-c7ca31df2449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:46.019086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a442e2-ad7c-4622-bfca-c7ca31df2449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:46.019959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a442e2-ad7c-4622-bfca-c7ca31df2449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:46.021765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a442e2-ad7c-4622-bfca-c7ca31df2449 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:46.044329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1051/2081] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:46.044502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:46.057508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222122a-1a8a-4a89-925f-02a80d275910 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:46.063602 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:46.064286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:46.064651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:46.064935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] PATH_INFO: `/v3/projects/227915ca681d49028988693c81019dcc/users/f8ac22c8c242451b8fb506feb3a2502f/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:46.066966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227915ca681d49028988693c81019dcc, user_id=f8ac22c8c242451b8fb506feb3a2502f, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:46.070261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:46.083442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1031/2082] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:46 2022] PUT /identity/v3/projects/227915ca681d49028988693c81019dcc/users/f8ac22c8c242451b8fb506feb3a2502f/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:46.094480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5162f6-f8a3-4eb3-9243-914601e468fd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:46.094775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5162f6-f8a3-4eb3-9243-914601e468fd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:46.095055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5162f6-f8a3-4eb3-9243-914601e468fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:46.140912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ee5162f6-f8a3-4eb3-9243-914601e468fd None None] MFA Rules not processed for user `f8ac22c8c242451b8fb506feb3a2502f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:46.146116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ee5162f6-f8a3-4eb3-9243-914601e468fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:46.318706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1052/2083] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:46.329893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:46.333468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:46.466665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388a3dd-57f3-42e9-8201-15fe50c27d86 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:46.472331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c2d05-b42d-47b4-b1a6-5c1197b8d6d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:46.473479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c2d05-b42d-47b4-b1a6-5c1197b8d6d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:46.474046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c2d05-b42d-47b4-b1a6-5c1197b8d6d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:46.474590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c2d05-b42d-47b4-b1a6-5c1197b8d6d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:46.490976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c2d05-b42d-47b4-b1a6-5c1197b8d6d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:46.494227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c2d05-b42d-47b4-b1a6-5c1197b8d6d5 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:46.517058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1032/2084] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:47.336255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5162f6-f8a3-4eb3-9243-914601e468fd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:47.413697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5162f6-f8a3-4eb3-9243-914601e468fd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:47.418523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:47.419264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:47.419642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:47.423082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] PATH_INFO: `/v3/users/1615ebacf82f42daa5e6bbf8e1194030` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:47.423082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1615ebacf82f42daa5e6bbf8e1194030)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:47.424191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:47.427102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:47.538438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1615ebacf82f42daa5e6bbf8e1194030'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:47.576481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1615ebacf82f42daa5e6bbf8e1194030'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:47.602482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1615ebacf82f42daa5e6bbf8e1194030'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:47.616786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1615ebacf82f42daa5e6bbf8e1194030'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:47.617488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1615ebacf82f42daa5e6bbf8e1194030'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:47.628177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1053/2085] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:47 2022] DELETE /identity/v3/users/1615ebacf82f42daa5e6bbf8e1194030 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:47.631238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:47.631736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:47.635289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:47.696136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:47.697590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:47.711220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:47.782045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1033/2086] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:47.792966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:47.795178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:47.877852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678f6a-1180-495c-bcbd-cc3940ea7d40 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:47.882384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:47.883027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:47.883319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:47.883605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:47.895276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:47.896814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:47.907730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1054/2087] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:48.243282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:48.245033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:48.278211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6023f-cb41-40db-b395-92e7ad06a1c3 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:48.310236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:48.311111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:48.313122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:48.313575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] PATH_INFO: `/v3/projects/e3c753ea46ec4ef887997fbde7d53b5d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:48.323383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c753ea46ec4ef887997fbde7d53b5d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:48.326087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:48.447123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c753ea46ec4ef887997fbde7d53b5d', 'request_id': 'req-ee66c6a6-fbd1-4359-997d-d977224612b8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.463665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c753ea46ec4ef887997fbde7d53b5d', 'request_id': 'req-ee66c6a6-fbd1-4359-997d-d977224612b8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.478677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c753ea46ec4ef887997fbde7d53b5d', 'request_id': 'req-ee66c6a6-fbd1-4359-997d-d977224612b8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.499141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] The token cache is being invalidate because project e3c753ea46ec4ef887997fbde7d53b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:48.500494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.501414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.504482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1034/2088] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:48 2022] DELETE /identity/v3/projects/e3c753ea46ec4ef887997fbde7d53b5d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:48.508138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:48.533850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31ed6-fd1b-4a73-8d42-ba80379ab5f3 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:48.689627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:48.692576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:48.692983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:48.693419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] PATH_INFO: `/v3/users/f8ac22c8c242451b8fb506feb3a2502f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:48.697358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ac22c8c242451b8fb506feb3a2502f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:48.699503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:48.739707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:48.772970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ac22c8c242451b8fb506feb3a2502f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.784111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ac22c8c242451b8fb506feb3a2502f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.807935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ac22c8c242451b8fb506feb3a2502f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.819993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ac22c8c242451b8fb506feb3a2502f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.820750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ac22c8c242451b8fb506feb3a2502f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:48.829797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1055/2089] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:48 2022] DELETE /identity/v3/users/f8ac22c8c242451b8fb506feb3a2502f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:49.208895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:49.210479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:49.227038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66c6a6-fbd1-4359-997d-d977224612b8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:49.340051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:49.341011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:49.341210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:49.341491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] PATH_INFO: `/v3/users/d1e3dde3b1fd48f89f553ccf1dc76a13` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:49.347000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e3dde3b1fd48f89f553ccf1dc76a13)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:49.350457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:49.439789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:49.456622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e3dde3b1fd48f89f553ccf1dc76a13'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.458411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e3dde3b1fd48f89f553ccf1dc76a13'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.480118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e3dde3b1fd48f89f553ccf1dc76a13'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.488197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e3dde3b1fd48f89f553ccf1dc76a13'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.489097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e3dde3b1fd48f89f553ccf1dc76a13'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.516899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1035/2090] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:49 2022] DELETE /identity/v3/users/d1e3dde3b1fd48f89f553ccf1dc76a13 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:49.536522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:49.539134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:49.558090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62719d-062f-42ea-98d4-b8e4a1df4438 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:49.645680 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:49.646951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:49.647715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:49.647839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] PATH_INFO: `/v3/projects/227915ca681d49028988693c81019dcc` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:49.650349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=227915ca681d49028988693c81019dcc)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:49.653160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:49.758270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227915ca681d49028988693c81019dcc', 'request_id': 'req-822e343e-7ad2-4296-8aa0-303405c70d95'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.769777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:49.772168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:49.774343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227915ca681d49028988693c81019dcc', 'request_id': 'req-822e343e-7ad2-4296-8aa0-303405c70d95'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.782115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227915ca681d49028988693c81019dcc', 'request_id': 'req-822e343e-7ad2-4296-8aa0-303405c70d95'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.803551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d76450-42f3-4c81-88a7-3c3062e92976 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:49.810652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] The token cache is being invalidate because project 227915ca681d49028988693c81019dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:49.810767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.813158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-822e343e-7ad2-4296-8aa0-303405c70d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:49.819998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1056/2091] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:49 2022] DELETE /identity/v3/projects/227915ca681d49028988693c81019dcc => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:49.824984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef5264-9752-4e04-ba7d-cb9b7f6105ef None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:49.825282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef5264-9752-4e04-ba7d-cb9b7f6105ef None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:49.825565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef5264-9752-4e04-ba7d-cb9b7f6105ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:49.862413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-85ef5264-9752-4e04-ba7d-cb9b7f6105ef None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:49.865644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-85ef5264-9752-4e04-ba7d-cb9b7f6105ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:49.905314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:49.905571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:49.906723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:49.907365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:49.908927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] PATH_INFO: `/v3/projects/5ed8bbfcf5ec4f74baf644f4a9d5f200` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:49.911569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed8bbfcf5ec4f74baf644f4a9d5f200)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:49.914304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:49.961134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1057/2092] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:49.970435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c806645-c893-4dbb-ae49-dfbe7bfb5355 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:49.970731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c806645-c893-4dbb-ae49-dfbe7bfb5355 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:49.971015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c806645-c893-4dbb-ae49-dfbe7bfb5355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:49.994472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3c806645-c893-4dbb-ae49-dfbe7bfb5355 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:49.997994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3c806645-c893-4dbb-ae49-dfbe7bfb5355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.008109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed8bbfcf5ec4f74baf644f4a9d5f200', 'request_id': 'req-2983c73f-ef3f-42a3-a783-fd785d1b56c4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:50.015932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed8bbfcf5ec4f74baf644f4a9d5f200', 'request_id': 'req-2983c73f-ef3f-42a3-a783-fd785d1b56c4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:50.028013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed8bbfcf5ec4f74baf644f4a9d5f200', 'request_id': 'req-2983c73f-ef3f-42a3-a783-fd785d1b56c4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:50.049029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] The token cache is being invalidate because project 5ed8bbfcf5ec4f74baf644f4a9d5f200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:50.049425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:50.050550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-2983c73f-ef3f-42a3-a783-fd785d1b56c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:50.053269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1036/2093] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:49 2022] DELETE /identity/v3/projects/5ed8bbfcf5ec4f74baf644f4a9d5f200 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:50.058550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d52f5c-a5d0-4aa3-8c1b-acba77995be8 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.058950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d52f5c-a5d0-4aa3-8c1b-acba77995be8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.059344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d52f5c-a5d0-4aa3-8c1b-acba77995be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.083348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-37d52f5c-a5d0-4aa3-8c1b-acba77995be8 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:50.084863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-37d52f5c-a5d0-4aa3-8c1b-acba77995be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.091343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1058/2094] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:50.097401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c806645-c893-4dbb-ae49-dfbe7bfb5355 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.105274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1037/2095] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:50.107932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d725ba1-5f54-462f-ba42-629a99375b16 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.108657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d725ba1-5f54-462f-ba42-629a99375b16 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.109176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d725ba1-5f54-462f-ba42-629a99375b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.111460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c806645-c893-4dbb-ae49-dfbe7bfb5355 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.117919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.118553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.118904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.119222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.119978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:50.120361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.126771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.130877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3d725ba1-5f54-462f-ba42-629a99375b16 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:50.131842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3d725ba1-5f54-462f-ba42-629a99375b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.144163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1059/2096] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:50.145137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1038/2097] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:50.149461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.151391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d725ba1-5f54-462f-ba42-629a99375b16 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.163674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d725ba1-5f54-462f-ba42-629a99375b16 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.165475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bc724-3a4f-4fd4-b0fe-6ed7fdd0b20e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.169884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.170851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.171213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.171404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.171524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.172136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.172243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:50.172441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.172606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.172960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.173747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.174738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.175880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.187185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1039/2098] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:50.191434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.202691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87866582-d72a-457b-a376-84c1920d56b3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.208786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.209631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.210233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.210233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.211038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.213425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.213582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1060/2099] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:50.217335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.239854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:50.253374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1040/2100] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:50.262862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.289461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c952b71-6615-465b-bf38-24be7245051a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.337686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748e83d-78e4-46ea-bc52-cedb38a199ae admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.339717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:50.344045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.346043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.346249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.346683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.347696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.349994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.355121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.355822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.356277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.356653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.357382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.359671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.385839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1061/2101] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:50.387594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1041/2102] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:50.392365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.393602 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.402944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ddb7b-6ac6-49a2-b97f-1591f69bdfc5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.404069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927f0bc-06c4-4411-955f-b368dccb56c0 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.408456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.409440 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.409726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.410001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.410133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.410695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.410965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.411351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.411771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.412431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.412700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.414425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.424450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1062/2103] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:50.428099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.432359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1042/2104] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:50.433541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.439432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c686-7951-425f-b665-ee37e7f73903 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.444712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.445432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.445787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.446160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.446464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60153aaf-5a5b-49a8-b799-8116861c38f8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.446996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.449023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.454864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.454864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.454864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.454864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.455341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.455341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.468853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1043/2105] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:50.469256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1063/2106] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:50.474514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.476662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.486286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481db585-ef23-4191-8cd4-2571ad4a28ee admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.489143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307069d3-bd0c-42a6-8afc-d0ea6dab2386 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.493621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff6b5b-e328-4f5b-a4f1-9f4f048c8164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.494672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6b5b-e328-4f5b-a4f1-9f4f048c8164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.494795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6b5b-e328-4f5b-a4f1-9f4f048c8164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.495047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40d4c-d7f6-49f8-9f99-3eaa47752331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.495171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6b5b-e328-4f5b-a4f1-9f4f048c8164 admin admin] PATH_INFO: `/v3/projects/b68f4eb16e0c4d548c2ee3eb7b03f651/users/8558032ae6b947ad957a32aab4c90f3b/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.495953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40d4c-d7f6-49f8-9f99-3eaa47752331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.496387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40d4c-d7f6-49f8-9f99-3eaa47752331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.496762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40d4c-d7f6-49f8-9f99-3eaa47752331 admin admin] PATH_INFO: `/v3/projects/0ce2648260ac4473bc11225e9802b286/users/cb3d75688889448d956755880925069a/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.518690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff6b5b-e328-4f5b-a4f1-9f4f048c8164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68f4eb16e0c4d548c2ee3eb7b03f651, user_id=8558032ae6b947ad957a32aab4c90f3b, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.522269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff6b5b-e328-4f5b-a4f1-9f4f048c8164 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.531834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c40d4c-d7f6-49f8-9f99-3eaa47752331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce2648260ac4473bc11225e9802b286, user_id=cb3d75688889448d956755880925069a, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.544811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c40d4c-d7f6-49f8-9f99-3eaa47752331 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.567229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1044/2107] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:50 2022] PUT /identity/v3/projects/b68f4eb16e0c4d548c2ee3eb7b03f651/users/8558032ae6b947ad957a32aab4c90f3b/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:50.570181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1064/2108] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:50 2022] PUT /identity/v3/projects/0ce2648260ac4473bc11225e9802b286/users/cb3d75688889448d956755880925069a/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:50.578025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.578586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.579917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.581790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.582085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.582374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.615292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] MFA Rules not processed for user `cb3d75688889448d956755880925069a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:50.616532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.617464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] MFA Rules not processed for user `8558032ae6b947ad957a32aab4c90f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:50.619199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.723418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1065/2109] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:50.730912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.732369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.742525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1045/2110] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:50.755828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.757595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.809095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cb955-e8cf-40fd-b816-2f67e879fc76 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.813783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.814674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.815198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.815484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.816509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:50.832470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.834382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.843706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1066/2111] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:50.844512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:50.845822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:50.860285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ccc95-fa24-4dba-8a92-ed702217d1ab None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.866282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587093ee-ede3-429b-8c09-770fe78af7f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.867069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587093ee-ede3-429b-8c09-770fe78af7f9 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.867359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587093ee-ede3-429b-8c09-770fe78af7f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.867735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587093ee-ede3-429b-8c09-770fe78af7f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.874772 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:50.886867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587093ee-ede3-429b-8c09-770fe78af7f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.888905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587093ee-ede3-429b-8c09-770fe78af7f9 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:50.900678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1046/2112] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:50.948009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21371f-c270-468a-8aa2-f78c31ef7e2f service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:50.953911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:50.954633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:50.954990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:50.955338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] PATH_INFO: `/v3/users/b553ac3f41f24bcca3b6a3ae730f05e9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:50.956691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b553ac3f41f24bcca3b6a3ae730f05e9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:50.961161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:51.034793 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b553ac3f41f24bcca3b6a3ae730f05e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.046589 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b553ac3f41f24bcca3b6a3ae730f05e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.073211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b553ac3f41f24bcca3b6a3ae730f05e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.085929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b553ac3f41f24bcca3b6a3ae730f05e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.086583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b553ac3f41f24bcca3b6a3ae730f05e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.099895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:51.100966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:51.101372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:51.106241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1067/2113] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:50 2022] DELETE /identity/v3/users/b553ac3f41f24bcca3b6a3ae730f05e9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:51.140559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:51.141961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:51.244411 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1047/2114] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:51.271972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:51.274081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:51.360752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdce46b-6f6f-49df-b8a4-deadfaf41ab1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:51.364432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efe9cd-3d27-413c-b908-7c680332a5c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:51.365025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efe9cd-3d27-413c-b908-7c680332a5c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:51.365311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efe9cd-3d27-413c-b908-7c680332a5c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:51.365590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efe9cd-3d27-413c-b908-7c680332a5c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:51.375652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efe9cd-3d27-413c-b908-7c680332a5c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:51.377128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efe9cd-3d27-413c-b908-7c680332a5c2 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:51.394671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:51.395866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1068/2115] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:51.396336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:51.409717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afc8fa-bb26-45cd-abe7-8943cbd58058 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:51.415550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:51.416287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:51.416700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:51.417204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] PATH_INFO: `/v3/users/8558032ae6b947ad957a32aab4c90f3b` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:51.418829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8558032ae6b947ad957a32aab4c90f3b)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:51.421495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:51.476490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8558032ae6b947ad957a32aab4c90f3b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.477298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8558032ae6b947ad957a32aab4c90f3b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.492055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8558032ae6b947ad957a32aab4c90f3b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.501437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8558032ae6b947ad957a32aab4c90f3b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.502574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8558032ae6b947ad957a32aab4c90f3b'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.520051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1048/2116] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:51 2022] DELETE /identity/v3/users/8558032ae6b947ad957a32aab4c90f3b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:51.524947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:51.525241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:51.525525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:51.546161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:51.547362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:51.618255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:51.640670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1069/2117] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:51.650244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:51.651711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:51.692268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:51.699541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:51.703026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:51.713664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6b59e-f98d-4a3c-8c82-16ac75157cc7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:51.719721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:51.720483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:51.720691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:51.720966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] PATH_INFO: `/v3/projects/a81e9c34f6a9482ca8a9b9965aded674` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:51.722134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a81e9c34f6a9482ca8a9b9965aded674)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:51.724522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:51.734530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f6416-a528-487b-851d-97bada74f9bc admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:51.739096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:51.739959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:51.740287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:51.740685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:51.751678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:51.753986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:51.764768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1049/2118] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:51.780858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a81e9c34f6a9482ca8a9b9965aded674', 'request_id': 'req-a639c56f-e758-406b-a144-1d1e839f9769'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.789513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a81e9c34f6a9482ca8a9b9965aded674', 'request_id': 'req-a639c56f-e758-406b-a144-1d1e839f9769'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.797418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a81e9c34f6a9482ca8a9b9965aded674', 'request_id': 'req-a639c56f-e758-406b-a144-1d1e839f9769'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.816919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] The token cache is being invalidate because project a81e9c34f6a9482ca8a9b9965aded674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:51.817385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.818068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.819613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1070/2119] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:51 2022] DELETE /identity/v3/projects/a81e9c34f6a9482ca8a9b9965aded674 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:51.823837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:51.834532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80543995-caa8-4403-96f6-f49dfb645625 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:51.895386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:51.896399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:51.896399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:51.896643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] PATH_INFO: `/v3/users/52ebef836f4a424bb92c325fb0db81da` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:51.897778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ebef836f4a424bb92c325fb0db81da)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:51.899979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:51.954731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ebef836f4a424bb92c325fb0db81da'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.955535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ebef836f4a424bb92c325fb0db81da'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.968080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ebef836f4a424bb92c325fb0db81da'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.976229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ebef836f4a424bb92c325fb0db81da'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.977085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ebef836f4a424bb92c325fb0db81da'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:51.990905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1050/2120] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:51 2022] DELETE /identity/v3/users/52ebef836f4a424bb92c325fb0db81da => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:52.320789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:52.322293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:52.346373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639c56f-e758-406b-a144-1d1e839f9769 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:52.485933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:52.486961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.487682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.488174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] PATH_INFO: `/v3/projects/b68f4eb16e0c4d548c2ee3eb7b03f651` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.489845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68f4eb16e0c4d548c2ee3eb7b03f651)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:52.492042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:52.520001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:52.522661 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:52.553633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c6e48-ab63-4bf5-a567-1cc5b49acfef admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:52.573953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:52.575718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.576413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.577610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] PATH_INFO: `/v3/projects/6a5a329774f84244a82852e5bf985d1c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.579575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5a329774f84244a82852e5bf985d1c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:52.582270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:52.618628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68f4eb16e0c4d548c2ee3eb7b03f651', 'request_id': 'req-d8f59671-5f47-4bab-a60c-d06cf901fb21'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.637366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68f4eb16e0c4d548c2ee3eb7b03f651', 'request_id': 'req-d8f59671-5f47-4bab-a60c-d06cf901fb21'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.648650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68f4eb16e0c4d548c2ee3eb7b03f651', 'request_id': 'req-d8f59671-5f47-4bab-a60c-d06cf901fb21'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.672700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:52.679229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] The token cache is being invalidate because project b68f4eb16e0c4d548c2ee3eb7b03f651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:52.679868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.680560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:52.681189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8f59671-5f47-4bab-a60c-d06cf901fb21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.683028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5a329774f84244a82852e5bf985d1c', 'request_id': 'req-1a8dd1e1-be1c-4daf-81f1-0823885b303d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.683933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1071/2121] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:52 2022] DELETE /identity/v3/projects/b68f4eb16e0c4d548c2ee3eb7b03f651 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:52.690742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c8600-1877-4f83-94a0-d52a1d88e00a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.691289 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c8600-1877-4f83-94a0-d52a1d88e00a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.691767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c8600-1877-4f83-94a0-d52a1d88e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.699814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5a329774f84244a82852e5bf985d1c', 'request_id': 'req-1a8dd1e1-be1c-4daf-81f1-0823885b303d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.711764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5a329774f84244a82852e5bf985d1c', 'request_id': 'req-1a8dd1e1-be1c-4daf-81f1-0823885b303d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.721744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-741c8600-1877-4f83-94a0-d52a1d88e00a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:52.723414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-741c8600-1877-4f83-94a0-d52a1d88e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:52.737594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] The token cache is being invalidate because project 6a5a329774f84244a82852e5bf985d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:52.737894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.738696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1a8dd1e1-be1c-4daf-81f1-0823885b303d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:52.740432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1051/2122] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:52 2022] DELETE /identity/v3/projects/6a5a329774f84244a82852e5bf985d1c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:52.745177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f1895-3b71-4a52-9fed-c1b08534f702 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.745720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f1895-3b71-4a52-9fed-c1b08534f702 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.746087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f1895-3b71-4a52-9fed-c1b08534f702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.768928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-0c5f1895-3b71-4a52-9fed-c1b08534f702 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:52.770154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0c5f1895-3b71-4a52-9fed-c1b08534f702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:52.803453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1072/2123] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:52.813382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16e949-dc3a-40bd-9041-b1d16b4aeb4c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.813672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16e949-dc3a-40bd-9041-b1d16b4aeb4c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.813967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16e949-dc3a-40bd-9041-b1d16b4aeb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.834702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0a16e949-dc3a-40bd-9041-b1d16b4aeb4c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:52.835931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0a16e949-dc3a-40bd-9041-b1d16b4aeb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:52.846433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1073/2124] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:52.846829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1052/2125] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:52.854836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e705c0f-763a-4367-a522-bcd5c53ee727 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.855300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e705c0f-763a-4367-a522-bcd5c53ee727 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.855586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e705c0f-763a-4367-a522-bcd5c53ee727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.862040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f1895-3b71-4a52-9fed-c1b08534f702 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:52.875968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f1895-3b71-4a52-9fed-c1b08534f702 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:52.883738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:52.885010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.885422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.886113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.887630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:52.887773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7e705c0f-763a-4367-a522-bcd5c53ee727 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:52.888106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:52.889067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7e705c0f-763a-4367-a522-bcd5c53ee727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:52.890621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:52.900671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1074/2126] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:52.904100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e705c0f-763a-4367-a522-bcd5c53ee727 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:52.906722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1053/2127] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:52.911196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:52.914785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e705c0f-763a-4367-a522-bcd5c53ee727 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:52.920005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:52.920182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.920430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.921282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.922098 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:27:52.922404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:52.923995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:52.925255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770af9cf-ea59-4fff-8179-6466fd7d5c29 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:52.930038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:52.930660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.930949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.931642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.932439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:52.934409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:52.937367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1075/2128] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:27:52.941683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:52.954487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d6eb5-70dd-4156-8afa-3527e689c9a3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:52.959903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:52.960558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:52.960900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:52.961328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:52.962097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:52.964344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:52.973289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1054/2129] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:52 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:52.984847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.001893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1076/2130] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:52 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:53.005604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.066911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49712a0-57f7-479c-a39f-db21c7c6cf68 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.072884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.073692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.074694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.074977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.077421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.079711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d1274-ec1f-417a-a414-f5ea1617bbc6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.082087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.086433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.087136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.087429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.087710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.088517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.090873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.114584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1055/2131] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:52 2022] POST /identity/v3/users => generated 305 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:53.116944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.117133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1077/2132] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:53 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:53.121148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.130267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0cd7-0cc4-4865-bbbe-70b439973bae admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.131586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda1d24-55e1-436c-b555-d6c19d4590c9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.134957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.135580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.135871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.136155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.136890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.137096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.137703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.137990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.138299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.138471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.139024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.141350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.150869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1056/2133] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:53.151947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1078/2134] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:27:53.156672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.156960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.167528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83aecd-0e5d-46be-a6d1-a848b1420717 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.168093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc867582-79e4-47fd-89c3-3e7e9b9c6bc4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.175156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.175865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.176063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.176212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.176374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.176715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.177081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.177198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.177295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.178186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.178744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.179766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.190928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1079/2135] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:53.196015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.197132 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1057/2136] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:53.197701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.209671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7248ecd-232e-420e-b0bd-8789c1cfb5cb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.210635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785afb4-18cd-4c04-95ae-e38ed972dc21 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.214490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0954d3f-3312-460e-b8b4-ab29e4926c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.215163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954d3f-3312-460e-b8b4-ab29e4926c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.215442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954d3f-3312-460e-b8b4-ab29e4926c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.215719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954d3f-3312-460e-b8b4-ab29e4926c91 admin admin] PATH_INFO: `/v3/projects/c9132e81d9f54fb69f5f3707008b0b43/users/55b6497e41c743e595b1b382af9e29f9/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.217036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c76d26-ea7e-4eab-aeaa-2619ecf7eef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.218829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c76d26-ea7e-4eab-aeaa-2619ecf7eef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.219183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c76d26-ea7e-4eab-aeaa-2619ecf7eef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.219474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c76d26-ea7e-4eab-aeaa-2619ecf7eef9 admin admin] PATH_INFO: `/v3/projects/0c437939c9f048cca5eaac0d883c4780/users/95444302d9384587a921c66a8d900edf/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.220931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:53.235750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0954d3f-3312-460e-b8b4-ab29e4926c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9132e81d9f54fb69f5f3707008b0b43, user_id=55b6497e41c743e595b1b382af9e29f9, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.237916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c76d26-ea7e-4eab-aeaa-2619ecf7eef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c437939c9f048cca5eaac0d883c4780, user_id=95444302d9384587a921c66a8d900edf, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.238824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0954d3f-3312-460e-b8b4-ab29e4926c91 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.240602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c76d26-ea7e-4eab-aeaa-2619ecf7eef9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.242798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:53.255537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1058/2137] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:53 2022] PUT /identity/v3/projects/c9132e81d9f54fb69f5f3707008b0b43/users/55b6497e41c743e595b1b382af9e29f9/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:53.261905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1080/2138] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:53 2022] PUT /identity/v3/projects/0c437939c9f048cca5eaac0d883c4780/users/95444302d9384587a921c66a8d900edf/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:53.267305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803083e9-490b-476a-8508-947d1894a675 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.267670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803083e9-490b-476a-8508-947d1894a675 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.268073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803083e9-490b-476a-8508-947d1894a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.272443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.273331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.273616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.297419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-803083e9-490b-476a-8508-947d1894a675 None None] MFA Rules not processed for user `55b6497e41c743e595b1b382af9e29f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:53.299010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-803083e9-490b-476a-8508-947d1894a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:53.301439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] MFA Rules not processed for user `95444302d9384587a921c66a8d900edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:53.302632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:53.368711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1059/2139] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:53.373168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803083e9-490b-476a-8508-947d1894a675 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.374866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1081/2140] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:53.387139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.388650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:53.440630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803083e9-490b-476a-8508-947d1894a675 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.445881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.446546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.446818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.447116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.447819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.449809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.478959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c8f3-5f13-4417-b1b4-03fce614cc7e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.487476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4b8f7545934151aafa986da871b944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.488107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.488393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.488735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.494814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1060/2141] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:27:53 2022] POST /identity/v3/projects => generated 407 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:27:53.497418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.499021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.500798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.512021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1082/2142] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:27:53.564314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deaf28-034b-4808-b971-3592c44c9d8b admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.570044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.570771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.571247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.571802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.573323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.576303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.586971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:53.610112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1061/2143] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:27:53 2022] POST /identity/v3/users => generated 310 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:27:53.615118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.627682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cdaab-cb82-4d55-9e8d-8bccc79c4089 service glance] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.633287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.633936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.634224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.634514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.635196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.639240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.651594 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1083/2144] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:53.657185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.675697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d40ae1-1505-4197-83db-b310b4319732 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.684185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.684914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.685198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.685480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] PATH_INFO: `/v3/projects/393eaf5f303444108bb315f0d45198d4/users/a0ebbd3255f84fce903bc9d1e777dff6/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.752641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=393eaf5f303444108bb315f0d45198d4, user_id=a0ebbd3255f84fce903bc9d1e777dff6, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.755649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.791040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1062/2145] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:53 2022] PUT /identity/v3/projects/393eaf5f303444108bb315f0d45198d4/users/a0ebbd3255f84fce903bc9d1e777dff6/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:53.794475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.815020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1897d-e443-453f-934f-fa1fd8bb47aa admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.889066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:53.899183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df77aad-3a63-49cd-83d8-0eb271642d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.899817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df77aad-3a63-49cd-83d8-0eb271642d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.900097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df77aad-3a63-49cd-83d8-0eb271642d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.900504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df77aad-3a63-49cd-83d8-0eb271642d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.901243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df77aad-3a63-49cd-83d8-0eb271642d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.902997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df77aad-3a63-49cd-83d8-0eb271642d4a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.915783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1084/2146] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:27:53.920030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:53.936252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079af1f-6ae8-47cc-bfa8-c1ce44bb1675 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:53.941922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:53.942780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.943170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.943555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] PATH_INFO: `/v3/projects/393eaf5f303444108bb315f0d45198d4/users/a0ebbd3255f84fce903bc9d1e777dff6/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:53.945728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=393eaf5f303444108bb315f0d45198d4, user_id=a0ebbd3255f84fce903bc9d1e777dff6, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:53.948663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:53.960900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1063/2147] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:27:53 2022] PUT /identity/v3/projects/393eaf5f303444108bb315f0d45198d4/users/a0ebbd3255f84fce903bc9d1e777dff6/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:27:53.972882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:53.973171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:53.973493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:54.002915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] MFA Rules not processed for user `a0ebbd3255f84fce903bc9d1e777dff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:54.004120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:54.088143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1085/2148] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:53 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:27:54.102305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:54.104130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:54.183575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dd2b1-49b4-4b10-b6db-c608f3e105c4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:54.188461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:54.189257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:54.189528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:54.189858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:54.199979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:54.201442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:54.212633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1064/2149] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:54 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:54.259434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:54.261243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:54.331570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9361451-37d4-4f48-ba7a-48279542e358 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:54.337117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:54.338010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:54.338327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:54.338727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] PATH_INFO: `/v3/users/cb3d75688889448d956755880925069a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:54.340198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3d75688889448d956755880925069a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:54.342256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:54.384386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:54.385913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:54.396412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3d75688889448d956755880925069a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:54.399754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013c01a-5ff0-4f3b-afbe-10fc98f638e1 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:54.403732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:54.404344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:54.404837 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:54.406043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:54.406729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3d75688889448d956755880925069a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:54.421574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3d75688889448d956755880925069a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:54.425483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:54.432333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3d75688889448d956755880925069a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:54.433155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2717bb17-ff3a-46c0-8f8e-e45397c31dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3d75688889448d956755880925069a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:54.441849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1086/2150] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:27:54 2022] DELETE /identity/v3/users/cb3d75688889448d956755880925069a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:27:54.446385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:54.446681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:54.446962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:54.466857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:27:54.468305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:54.475429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:54.476941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:54.486607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1065/2151] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:54.529444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1087/2152] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:27:54.539742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:54.541208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:54.601945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b941fc9-dbe6-4432-81fe-6def271cfd38 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:54.610165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:27:54.672133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:54.672974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:54.673338 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:54.673712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:54.685460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:54.687913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:54.699989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1066/2153] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:27:55.065307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:27:55.066847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:27:55.080426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf63f0-4f65-47fa-8c4e-c1db52155716 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:27:55.087201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:27:55.087990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:27:55.088349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:27:55.088860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] PATH_INFO: `/v3/projects/0ce2648260ac4473bc11225e9802b286` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:27:55.090113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce2648260ac4473bc11225e9802b286)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:27:55.104172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:27:55.162626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce2648260ac4473bc11225e9802b286', 'request_id': 'req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:55.169905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce2648260ac4473bc11225e9802b286', 'request_id': 'req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:55.180579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce2648260ac4473bc11225e9802b286', 'request_id': 'req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:55.195603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] The token cache is being invalidate because project 0ce2648260ac4473bc11225e9802b286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:27:55.195896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:55.196593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b9ab3c3d-12b2-4c99-9eaf-a32a3d50c81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:27:55.198120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1088/2154] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:27:55 2022] DELETE /identity/v3/projects/0ce2648260ac4473bc11225e9802b286 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:04.056236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:04.057651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:04.070382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0d560-438c-4e62-b7fc-7142debbb476 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:04.136480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:04.137333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:04.137619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:04.137896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] PATH_INFO: `/v3/users/55b6497e41c743e595b1b382af9e29f9` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:04.140013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b6497e41c743e595b1b382af9e29f9)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:04.142421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:04.212654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b6497e41c743e595b1b382af9e29f9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:04.213368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b6497e41c743e595b1b382af9e29f9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:04.230986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b6497e41c743e595b1b382af9e29f9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:04.239656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b6497e41c743e595b1b382af9e29f9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:04.240522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b6497e41c743e595b1b382af9e29f9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:04.260903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1067/2155] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:04 2022] DELETE /identity/v3/users/55b6497e41c743e595b1b382af9e29f9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:04.266622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:04.267178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:04.267178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:04.293661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:04.294889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:04.362795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1089/2156] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:04.373051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:04.374654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:04.412647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:04.443933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51345e1-654f-49a2-adc3-25dd95e8900d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:04.447710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:04.448373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:04.448702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:04.448981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:04.459321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:04.460781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:04.472253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1068/2157] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:04.978476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:04.979930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:04.992488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03d1d1-1aad-4bca-a134-d9b5be0a1f78 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:05.007006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:05.007696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:05.013671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:05.014103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] PATH_INFO: `/v3/projects/c9132e81d9f54fb69f5f3707008b0b43` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:05.015644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9132e81d9f54fb69f5f3707008b0b43)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:05.017796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:05.039432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:05.084441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9132e81d9f54fb69f5f3707008b0b43', 'request_id': 'req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.095008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9132e81d9f54fb69f5f3707008b0b43', 'request_id': 'req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.102616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9132e81d9f54fb69f5f3707008b0b43', 'request_id': 'req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.116867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] The token cache is being invalidate because project c9132e81d9f54fb69f5f3707008b0b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:05.117367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.118389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.120521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1090/2158] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:04 2022] DELETE /identity/v3/projects/c9132e81d9f54fb69f5f3707008b0b43 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:05.125138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:05.138091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65d35b-6efe-4286-b7b3-fddf6e190c7b service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:05.220558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:05.221229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:05.221513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:05.221800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] PATH_INFO: `/v3/users/a0ebbd3255f84fce903bc9d1e777dff6` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:05.223150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ebbd3255f84fce903bc9d1e777dff6)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:05.225206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:05.301996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ebbd3255f84fce903bc9d1e777dff6'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.302823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ebbd3255f84fce903bc9d1e777dff6'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.321875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ebbd3255f84fce903bc9d1e777dff6'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.330440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ebbd3255f84fce903bc9d1e777dff6'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.331261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-68011cfc-ac58-4536-9feb-0579016017cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ebbd3255f84fce903bc9d1e777dff6'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.346165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1069/2159] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:05 2022] DELETE /identity/v3/users/a0ebbd3255f84fce903bc9d1e777dff6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:05.647056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:05.648724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:05.662875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c986dc-fdc1-4c82-b67c-0bab4ba7919e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:05.745668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:05.746416 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:05.746803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:05.747150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] PATH_INFO: `/v3/projects/393eaf5f303444108bb315f0d45198d4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:05.748849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=393eaf5f303444108bb315f0d45198d4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:05.751405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:05.814240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393eaf5f303444108bb315f0d45198d4', 'request_id': 'req-751515d2-9054-4579-9bd3-a36efb6b9474'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.822605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393eaf5f303444108bb315f0d45198d4', 'request_id': 'req-751515d2-9054-4579-9bd3-a36efb6b9474'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.830814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393eaf5f303444108bb315f0d45198d4', 'request_id': 'req-751515d2-9054-4579-9bd3-a36efb6b9474'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.848386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] The token cache is being invalidate because project 393eaf5f303444108bb315f0d45198d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:05.848945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.851041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-751515d2-9054-4579-9bd3-a36efb6b9474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:05.852587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1091/2160] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:05 2022] DELETE /identity/v3/projects/393eaf5f303444108bb315f0d45198d4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:05.858465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766c1f-ce98-4668-aa77-7d469bcf4908 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:05.859258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766c1f-ce98-4668-aa77-7d469bcf4908 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:05.859626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766c1f-ce98-4668-aa77-7d469bcf4908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:05.881818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ce766c1f-ce98-4668-aa77-7d469bcf4908 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:05.883310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ce766c1f-ce98-4668-aa77-7d469bcf4908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:05.934625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:05.950582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1070/2161] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:05.959135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020091b-cc36-47fa-9dbf-1bf6575fd708 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:05.959429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020091b-cc36-47fa-9dbf-1bf6575fd708 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:05.959709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020091b-cc36-47fa-9dbf-1bf6575fd708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:05.980205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f020091b-cc36-47fa-9dbf-1bf6575fd708 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:05.981432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f020091b-cc36-47fa-9dbf-1bf6575fd708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:05.989944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1092/2162] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:05.994179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766c1f-ce98-4668-aa77-7d469bcf4908 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.004053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766c1f-ce98-4668-aa77-7d469bcf4908 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.008710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.009325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.009612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.009894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.010584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:28:06.010923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.012515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.024442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1071/2163] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:28:06.028511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020091b-cc36-47fa-9dbf-1bf6575fd708 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.038976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020091b-cc36-47fa-9dbf-1bf6575fd708 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.043703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.044336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.044639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.044915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.046454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.048348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.075783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1093/2164] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:06 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:06.080734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.091329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2afa-f1c4-4b37-99e6-d8dd17e1e275 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.150988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.151676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.151872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.152145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.153278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.155793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.180973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1072/2165] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:06 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:06.185228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.195943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:06.204024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07b1ea-ac30-4da1-9819-7fa65f4f520d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.211379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.212350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.212672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.212954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.213972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.215597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.229768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1094/2166] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:28:06.230926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.242172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecef4d-32c1-497a-8913-94dda18dc348 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.247181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14818f5b-b9c2-45fe-8653-c1d5b311c572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.247812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14818f5b-b9c2-45fe-8653-c1d5b311c572 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.248680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14818f5b-b9c2-45fe-8653-c1d5b311c572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.248980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14818f5b-b9c2-45fe-8653-c1d5b311c572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.249707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14818f5b-b9c2-45fe-8653-c1d5b311c572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.251372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14818f5b-b9c2-45fe-8653-c1d5b311c572 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.267971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1073/2167] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:28:06.269329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.279862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1e93-fb6a-44a4-97b8-4bae04bbc7c9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.284476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb7d03-4ffb-4c3e-b859-ed9dbe6d3ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.285172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7d03-4ffb-4c3e-b859-ed9dbe6d3ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.285457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7d03-4ffb-4c3e-b859-ed9dbe6d3ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.285732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7d03-4ffb-4c3e-b859-ed9dbe6d3ee7 admin admin] PATH_INFO: `/v3/projects/2390f6fc6fcc489ea09d5647b821e780/users/e9f5064ae07b4892bc0c4dba43f26412/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.301664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb7d03-4ffb-4c3e-b859-ed9dbe6d3ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2390f6fc6fcc489ea09d5647b821e780, user_id=e9f5064ae07b4892bc0c4dba43f26412, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.312089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb7d03-4ffb-4c3e-b859-ed9dbe6d3ee7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.325349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1095/2168] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:06 2022] PUT /identity/v3/projects/2390f6fc6fcc489ea09d5647b821e780/users/e9f5064ae07b4892bc0c4dba43f26412/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:06.336754 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900db856-acb8-424b-b7c3-da6c38076645 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.337051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900db856-acb8-424b-b7c3-da6c38076645 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.337331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900db856-acb8-424b-b7c3-da6c38076645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.365439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-900db856-acb8-424b-b7c3-da6c38076645 None None] MFA Rules not processed for user `e9f5064ae07b4892bc0c4dba43f26412`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:06.366649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-900db856-acb8-424b-b7c3-da6c38076645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:06.431187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb1d54-529b-433e-9410-9ed5000767ac None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.431451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb1d54-529b-433e-9410-9ed5000767ac None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.431698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb1d54-529b-433e-9410-9ed5000767ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.441237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1074/2169] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:28:06.445859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900db856-acb8-424b-b7c3-da6c38076645 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.450554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8ddb1d54-529b-433e-9410-9ed5000767ac None None] MFA Rules not processed for user `55bb264bd9834f5fa61b58362ec48d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:06.452668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8ddb1d54-529b-433e-9410-9ed5000767ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:06.502310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:06.518317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1096/2170] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:28:06.521452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900db856-acb8-424b-b7c3-da6c38076645 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.526523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.527181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.527531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.527850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.528407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.530492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.559428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1075/2171] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:06 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:28:06.563970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb1d54-529b-433e-9410-9ed5000767ac None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.575693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb1d54-529b-433e-9410-9ed5000767ac None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.640672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.641301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.641584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.641861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.643017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.644431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.669860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1097/2172] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:06 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:06.671685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.683271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec111c-b348-4c75-8c11-234105ab4939 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.688458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.689253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.689622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.689908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.690612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.692247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.704652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1076/2173] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:06.709536 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.720167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc826-b922-40ff-9deb-42d42c808764 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.725243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.725914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.726193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.726470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] PATH_INFO: `/v3/projects/52f630e1407c407683f633c4ec6f2b61/users/d996575227644f88b85f9b45317c1dd8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.745818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f630e1407c407683f633c4ec6f2b61, user_id=d996575227644f88b85f9b45317c1dd8, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.749426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.764404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1098/2174] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:06 2022] PUT /identity/v3/projects/52f630e1407c407683f633c4ec6f2b61/users/d996575227644f88b85f9b45317c1dd8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:06.768319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.780164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712bc89-b757-4e83-a712-917f57c67d92 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.846211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab4c2f-b4fe-4303-b6d0-b51e966c96ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.847126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4c2f-b4fe-4303-b6d0-b51e966c96ac admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.847246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4c2f-b4fe-4303-b6d0-b51e966c96ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.847691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4c2f-b4fe-4303-b6d0-b51e966c96ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.848168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab4c2f-b4fe-4303-b6d0-b51e966c96ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.849823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab4c2f-b4fe-4303-b6d0-b51e966c96ac admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.869733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1077/2175] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:06.873944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:06.885227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa3974-7872-435b-be68-70439e8ae37a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:06.891368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:06.892237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.892521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.892833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] PATH_INFO: `/v3/projects/52f630e1407c407683f633c4ec6f2b61/users/d996575227644f88b85f9b45317c1dd8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.900927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f630e1407c407683f633c4ec6f2b61, user_id=d996575227644f88b85f9b45317c1dd8, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:06.903215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:06.921981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1099/2176] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:06 2022] PUT /identity/v3/projects/52f630e1407c407683f633c4ec6f2b61/users/d996575227644f88b85f9b45317c1dd8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:06.933641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:06.933936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:06.934221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:06.970546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] MFA Rules not processed for user `d996575227644f88b85f9b45317c1dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:06.971476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:07.063058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1078/2177] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:07.069977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:07.071949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:07.125017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:07.164844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7bd2-bc65-4951-a33a-4ddb351d1351 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:07.168801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727e3e9-725d-4462-beab-283eaa0d0927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:07.169515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e3e9-725d-4462-beab-283eaa0d0927 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:07.169841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e3e9-725d-4462-beab-283eaa0d0927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:07.170700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e3e9-725d-4462-beab-283eaa0d0927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:07.181656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727e3e9-725d-4462-beab-283eaa0d0927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:07.183320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727e3e9-725d-4462-beab-283eaa0d0927 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:07.195902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1100/2178] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:07.349864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:07.351562 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:07.415628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:07.447799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f540c-bb12-42bc-8fcf-f57955e07f4e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:07.453320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:07.454006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:07.454852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:07.455290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] PATH_INFO: `/v3/users/1299be939f52480cad07e9d4773eddb7` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:07.456724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1299be939f52480cad07e9d4773eddb7)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:07.459305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:07.524614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1299be939f52480cad07e9d4773eddb7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.525447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1299be939f52480cad07e9d4773eddb7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.543568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1299be939f52480cad07e9d4773eddb7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.551936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1299be939f52480cad07e9d4773eddb7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.552809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1299be939f52480cad07e9d4773eddb7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.568104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1079/2179] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:07 2022] DELETE /identity/v3/users/1299be939f52480cad07e9d4773eddb7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:07.572927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:07.574025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:07.574817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:07.616974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:07.618219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:07.663242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:07.664707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:07.696010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1101/2180] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:28:07.706629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:07.708811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:07.752090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51853c8-490c-4e30-a589-8b869ef41e17 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:07.757508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:07.758436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:07.758876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:07.759346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] PATH_INFO: `/v3/users/e9f5064ae07b4892bc0c4dba43f26412` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:07.760638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f5064ae07b4892bc0c4dba43f26412)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:07.762582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:07.790814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7823d41-898d-42b2-8dfc-81d2d7e00582 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:07.797128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061fbaf-6c24-43ab-b8fb-af524ac381c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:07.798382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fbaf-6c24-43ab-b8fb-af524ac381c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:07.798382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fbaf-6c24-43ab-b8fb-af524ac381c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:07.798606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fbaf-6c24-43ab-b8fb-af524ac381c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:07.809191 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061fbaf-6c24-43ab-b8fb-af524ac381c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:07.811232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061fbaf-6c24-43ab-b8fb-af524ac381c5 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:07.819313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f5064ae07b4892bc0c4dba43f26412'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.820447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f5064ae07b4892bc0c4dba43f26412'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.824375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1102/2181] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:28:07.837509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f5064ae07b4892bc0c4dba43f26412'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.838924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:07.847912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f5064ae07b4892bc0c4dba43f26412'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.849291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f5064ae07b4892bc0c4dba43f26412'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:07.863758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1080/2182] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:07 2022] DELETE /identity/v3/users/e9f5064ae07b4892bc0c4dba43f26412 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:28:07.871981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:07.872277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:07.872587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:07.893872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:07.895227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:07.982029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1103/2183] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:08.004958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.006529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.079173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ecbba-2e27-402c-802a-67638bd63803 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.083347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa39f43-8139-45f3-935a-9703681f4514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.084008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa39f43-8139-45f3-935a-9703681f4514 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.084297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa39f43-8139-45f3-935a-9703681f4514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.084605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa39f43-8139-45f3-935a-9703681f4514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.094568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa39f43-8139-45f3-935a-9703681f4514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.096154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa39f43-8139-45f3-935a-9703681f4514 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.106180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1081/2184] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:08.186472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.188081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.229787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041abd69-0c51-4c53-ae9e-0108b0fc6501 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.248785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.249460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.249783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.250025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] PATH_INFO: `/v3/projects/f081895f333e4a4299a7ca4d6c9ffbe7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.251962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f081895f333e4a4299a7ca4d6c9ffbe7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.253934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.268180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:08.351339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f081895f333e4a4299a7ca4d6c9ffbe7', 'request_id': 'req-78012876-6ead-4a31-b7fe-25e1ac06cd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.362196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f081895f333e4a4299a7ca4d6c9ffbe7', 'request_id': 'req-78012876-6ead-4a31-b7fe-25e1ac06cd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.380156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f081895f333e4a4299a7ca4d6c9ffbe7', 'request_id': 'req-78012876-6ead-4a31-b7fe-25e1ac06cd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.401780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] The token cache is being invalidate because project f081895f333e4a4299a7ca4d6c9ffbe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:08.402218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.403504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-78012876-6ead-4a31-b7fe-25e1ac06cd2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.405063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1104/2185] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:08 2022] DELETE /identity/v3/projects/f081895f333e4a4299a7ca4d6c9ffbe7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:08.410914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17cac7-68a2-45a8-a6ca-a87d7506aa33 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.410914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17cac7-68a2-45a8-a6ca-a87d7506aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.410914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17cac7-68a2-45a8-a6ca-a87d7506aa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.430583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7f17cac7-68a2-45a8-a6ca-a87d7506aa33 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:08.432095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7f17cac7-68a2-45a8-a6ca-a87d7506aa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.508712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1082/2186] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:08.515930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da17a90-77db-4fbc-969f-fcc775eb86f7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.516429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da17a90-77db-4fbc-969f-fcc775eb86f7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.516905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da17a90-77db-4fbc-969f-fcc775eb86f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.537652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5da17a90-77db-4fbc-969f-fcc775eb86f7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:08.538868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5da17a90-77db-4fbc-969f-fcc775eb86f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.548726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1105/2187] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:08.553033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17cac7-68a2-45a8-a6ca-a87d7506aa33 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.562912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17cac7-68a2-45a8-a6ca-a87d7506aa33 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.568114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.568767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.569054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.569448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.570030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:28:08.570342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.571950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.589289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1083/2188] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:28:08.594927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da17a90-77db-4fbc-969f-fcc775eb86f7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.605792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da17a90-77db-4fbc-969f-fcc775eb86f7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.612895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.615054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.615215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.615907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.616282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.616728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.617910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.620742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.630898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c03a4a-7abe-4f49-9905-c6e5557e6eb0 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.636981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.637705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.637993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.638275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] PATH_INFO: `/v3/projects/2390f6fc6fcc489ea09d5647b821e780` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.639633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2390f6fc6fcc489ea09d5647b821e780)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.642196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.677555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1106/2189] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:08.681948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.716197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2390f6fc6fcc489ea09d5647b821e780', 'request_id': 'req-7017e906-70ac-4884-991f-383186f0c699'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.734775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2390f6fc6fcc489ea09d5647b821e780', 'request_id': 'req-7017e906-70ac-4884-991f-383186f0c699'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.744782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2390f6fc6fcc489ea09d5647b821e780', 'request_id': 'req-7017e906-70ac-4884-991f-383186f0c699'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.752259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:08.759697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] The token cache is being invalidate because project 2390f6fc6fcc489ea09d5647b821e780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:08.760230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.761581 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.764581 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1084/2190] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:08 2022] DELETE /identity/v3/projects/2390f6fc6fcc489ea09d5647b821e780 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:08.767971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.769964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.770156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3117-a6a8-43e3-99ea-12f52becfede admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.836633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.837259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.837541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.837939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.838602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.840503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.847044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017e906-70ac-4884-991f-383186f0c699 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.852722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.853484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.853835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.854118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] PATH_INFO: `/v3/users/d996575227644f88b85f9b45317c1dd8` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.855604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d996575227644f88b85f9b45317c1dd8)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.858202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.865703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1107/2191] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:08 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:28:08.869760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.871800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.880767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:08.885741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8515e-15a8-4b0e-8ea3-136d31ca7f61 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.893736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.894335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.895044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.895586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.896528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.898463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.912379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1108/2192] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:28:08.913441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.924434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd996575227644f88b85f9b45317c1dd8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.925616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd996575227644f88b85f9b45317c1dd8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.926335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4f121-c139-46e9-9789-70328da80ec7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:08.931532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:08.932157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:08.932448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:08.932762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:08.933911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:08.935167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:08.939809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd996575227644f88b85f9b45317c1dd8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.946844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1109/2193] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:08.949812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd996575227644f88b85f9b45317c1dd8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.950725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0820b885-c8ea-4827-a091-1435e9dca911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd996575227644f88b85f9b45317c1dd8'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:08.952231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:08.953546 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:08.968827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1085/2194] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:08 2022] DELETE /identity/v3/users/d996575227644f88b85f9b45317c1dd8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:08.969890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfcc89-4efa-4c65-a2ae-96fe98cc6761 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:09.077579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:09.078336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:09.078620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:09.078896 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] PATH_INFO: `/v3/projects/77d0443b289548b1b6e96bb268977c69/users/e8d0f88d70ba4646a10c89d9d32de574/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:09.096902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d0443b289548b1b6e96bb268977c69, user_id=e8d0f88d70ba4646a10c89d9d32de574, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:09.099563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:09.118352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1110/2195] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:08 2022] PUT /identity/v3/projects/77d0443b289548b1b6e96bb268977c69/users/e8d0f88d70ba4646a10c89d9d32de574/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:28:09.130425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:09.130719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:09.131001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:09.186865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] MFA Rules not processed for user `e8d0f88d70ba4646a10c89d9d32de574`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:09.188127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:09.254920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1086/2196] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:09.265755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:09.267554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:09.327919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f52b4-b454-4dd0-b20b-2d414c2e99b8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:09.331696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbb838-4def-4106-94ef-fe42dc2aa758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:09.332495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbb838-4def-4106-94ef-fe42dc2aa758 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:09.333087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbb838-4def-4106-94ef-fe42dc2aa758 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:09.333369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbb838-4def-4106-94ef-fe42dc2aa758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:09.343057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbb838-4def-4106-94ef-fe42dc2aa758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:09.344349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbb838-4def-4106-94ef-fe42dc2aa758 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:09.354522 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1111/2197] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:09.571300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:09.573080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:09.581797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:09.696948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fe1ea-8fd6-46a6-b202-b6b87bcc69ed None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:09.703890 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:09.704758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:09.705319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:09.705477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] PATH_INFO: `/v3/projects/52f630e1407c407683f633c4ec6f2b61` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:09.706814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f630e1407c407683f633c4ec6f2b61)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:09.709421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:09.794226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f630e1407c407683f633c4ec6f2b61', 'request_id': 'req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:09.807320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f630e1407c407683f633c4ec6f2b61', 'request_id': 'req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:09.823859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f630e1407c407683f633c4ec6f2b61', 'request_id': 'req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:09.838336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] The token cache is being invalidate because project 52f630e1407c407683f633c4ec6f2b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:09.838757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:09.839555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7e1597d6-7ed2-4cb9-a7f0-12fb05993dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:09.847685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1087/2198] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:09 2022] DELETE /identity/v3/projects/52f630e1407c407683f633c4ec6f2b61 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:09.855474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aed650-ee23-42f5-8d91-71605df6fbbd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:09.855963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aed650-ee23-42f5-8d91-71605df6fbbd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:09.856152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aed650-ee23-42f5-8d91-71605df6fbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:09.884249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-14aed650-ee23-42f5-8d91-71605df6fbbd None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:09.885305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-14aed650-ee23-42f5-8d91-71605df6fbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:09.951852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1112/2199] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:09.960066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6554-dcb2-469a-99cb-8b1d027edeb7 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:09.960442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6554-dcb2-469a-99cb-8b1d027edeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:09.960827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6554-dcb2-469a-99cb-8b1d027edeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:09.982036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-284c6554-dcb2-469a-99cb-8b1d027edeb7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:09.984095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-284c6554-dcb2-469a-99cb-8b1d027edeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:09.992188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1088/2200] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:09.996455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aed650-ee23-42f5-8d91-71605df6fbbd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.006823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aed650-ee23-42f5-8d91-71605df6fbbd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.011685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.012509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.012823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.013097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.014373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:28:10.014679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.016319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.018840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:10.030385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1113/2201] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:28:10.034510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6554-dcb2-469a-99cb-8b1d027edeb7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.044604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6554-dcb2-469a-99cb-8b1d027edeb7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.049604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.050222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.050506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.050782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.051446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.053457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.090547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1089/2202] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:10 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:10.095791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.111776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3bbf-1543-40d2-8b35-d79c9ccff15b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.180725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.181354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.181637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.181919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.182586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.184496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.206952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1114/2203] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:10 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:10.211221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.221943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2250fa-bc20-409e-b352-ded17496169a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.226642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.227261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.227545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.227826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.228524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.230152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.242342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1090/2204] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:10.246044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.256857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c87c6b-72fa-4761-a59a-34b1193c1086 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.261422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d242b9-9349-4931-9a82-09cbfa46678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.262028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d242b9-9349-4931-9a82-09cbfa46678d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.262309 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d242b9-9349-4931-9a82-09cbfa46678d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.262582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d242b9-9349-4931-9a82-09cbfa46678d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.263261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d242b9-9349-4931-9a82-09cbfa46678d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.264969 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d242b9-9349-4931-9a82-09cbfa46678d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.275506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1115/2205] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:10.279889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.290543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844e81-0e9e-418d-9a6f-1a3c61746eeb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.296139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.296882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.297172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.297460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] PATH_INFO: `/v3/projects/4dfa365749194607a175535aef6bdaec/users/2194b5a86fa14076ae40f63460e886fb/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.313551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dfa365749194607a175535aef6bdaec, user_id=2194b5a86fa14076ae40f63460e886fb, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.316229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.330930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1091/2206] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:10 2022] PUT /identity/v3/projects/4dfa365749194607a175535aef6bdaec/users/2194b5a86fa14076ae40f63460e886fb/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:10.341483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f139e-fde2-4692-b7c0-ccdd5763f52a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.342097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f139e-fde2-4692-b7c0-ccdd5763f52a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.342453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f139e-fde2-4692-b7c0-ccdd5763f52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.370822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9c5f139e-fde2-4692-b7c0-ccdd5763f52a None None] MFA Rules not processed for user `2194b5a86fa14076ae40f63460e886fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:10.372227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9c5f139e-fde2-4692-b7c0-ccdd5763f52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:10.441897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1116/2207] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:10.453708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.455442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:10.492467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:10.517450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db356e-a14f-4304-b69d-f921a9bfe462 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.521429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dacd8a-6f55-4948-bdda-106909f98d36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.522070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dacd8a-6f55-4948-bdda-106909f98d36 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.522357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dacd8a-6f55-4948-bdda-106909f98d36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.522638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dacd8a-6f55-4948-bdda-106909f98d36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.532977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dacd8a-6f55-4948-bdda-106909f98d36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.534433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dacd8a-6f55-4948-bdda-106909f98d36 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.544494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1092/2208] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:10.570018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f139e-fde2-4692-b7c0-ccdd5763f52a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.633870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f139e-fde2-4692-b7c0-ccdd5763f52a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.638563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.639705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.640128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.640417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] PATH_INFO: `/v3/users/2194b5a86fa14076ae40f63460e886fb` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.641643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2194b5a86fa14076ae40f63460e886fb)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.643564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.648143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:10.693749 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2194b5a86fa14076ae40f63460e886fb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:10.700693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2194b5a86fa14076ae40f63460e886fb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:10.712801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2194b5a86fa14076ae40f63460e886fb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:10.727756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2194b5a86fa14076ae40f63460e886fb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:10.728424 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2194b5a86fa14076ae40f63460e886fb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:10.738747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1117/2209] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:10 2022] DELETE /identity/v3/users/2194b5a86fa14076ae40f63460e886fb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:10.743204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce9e43-85f1-40ea-9ab2-0721cc859739 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.743639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce9e43-85f1-40ea-9ab2-0721cc859739 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.743798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce9e43-85f1-40ea-9ab2-0721cc859739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.764387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-afce9e43-85f1-40ea-9ab2-0721cc859739 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:10.765632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-afce9e43-85f1-40ea-9ab2-0721cc859739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:10.829808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1093/2210] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:10.839475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:10.841175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:10.882374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdd128-538d-4feb-9c12-570c491d8f4a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.882748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdd128-538d-4feb-9c12-570c491d8f4a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.883102 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdd128-538d-4feb-9c12-570c491d8f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.903898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2acdd128-538d-4feb-9c12-570c491d8f4a None None] MFA Rules not processed for user `e8d0f88d70ba4646a10c89d9d32de574`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:10.905869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2acdd128-538d-4feb-9c12-570c491d8f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:10.908406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646859f-acc1-4c3a-9eea-377e4695064d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:10.912077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:10.912731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:10.913065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:10.913347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:10.924788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:10.926264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:10.936886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1118/2211] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:10.964327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1094/2212] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:28:11.513732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:11.517885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:11.535534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ebd2a-db6c-47d0-ac6e-c06eb5943510 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:11.540349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:11.541173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:11.541712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:11.542076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] PATH_INFO: `/v3/projects/4dfa365749194607a175535aef6bdaec` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:11.544189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dfa365749194607a175535aef6bdaec)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:11.546614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:11.606504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dfa365749194607a175535aef6bdaec', 'request_id': 'req-0a9fd524-2063-412d-a18f-a3ce1c6c50be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:11.615126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dfa365749194607a175535aef6bdaec', 'request_id': 'req-0a9fd524-2063-412d-a18f-a3ce1c6c50be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:11.622222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dfa365749194607a175535aef6bdaec', 'request_id': 'req-0a9fd524-2063-412d-a18f-a3ce1c6c50be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:11.635270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] The token cache is being invalidate because project 4dfa365749194607a175535aef6bdaec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:11.635558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:11.636389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9fd524-2063-412d-a18f-a3ce1c6c50be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:11.638411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1119/2213] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:11 2022] DELETE /identity/v3/projects/4dfa365749194607a175535aef6bdaec => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:11.644003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9fbdb-40ba-49b1-8923-931b6296d24f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:11.644388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9fbdb-40ba-49b1-8923-931b6296d24f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:11.644840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9fbdb-40ba-49b1-8923-931b6296d24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:11.669175 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8be9fbdb-40ba-49b1-8923-931b6296d24f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:11.670457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8be9fbdb-40ba-49b1-8923-931b6296d24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:11.766823 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d32d4-44a1-49ed-ae8d-c46519c939cb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:11.767229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1095/2214] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:11.767986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d32d4-44a1-49ed-ae8d-c46519c939cb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:11.768302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d32d4-44a1-49ed-ae8d-c46519c939cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:11.794553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-929d32d4-44a1-49ed-ae8d-c46519c939cb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:11.796649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-929d32d4-44a1-49ed-ae8d-c46519c939cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:11.806269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1120/2215] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:11.811110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9fbdb-40ba-49b1-8923-931b6296d24f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:11.821762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9fbdb-40ba-49b1-8923-931b6296d24f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:11.832273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:11.833373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:11.833663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:11.834009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:11.834858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:28:11.835239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:11.838394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:11.880826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d32d4-44a1-49ed-ae8d-c46519c939cb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:11.884885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1096/2216] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:28:11.886185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:11.893471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d32d4-44a1-49ed-ae8d-c46519c939cb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:11.899711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:11.900919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:11.901391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:11.901879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:11.902762 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:11.905124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:11.943262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1121/2217] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:11 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:11.947800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:11.963339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2162706-998f-4294-a255-d015c9cb66f1 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:11.968634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:12.047129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:12.047766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:12.048048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:12.048326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:12.049029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:12.050941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:12.079552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1097/2218] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:11 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:12.080842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:12.094719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa157b86-2019-4aa6-b8d7-8667922b5047 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:12.101623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:12.102395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:12.102746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:12.103132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:12.104334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:12.106898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:12.118266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1122/2219] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:28:12.122451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:12.133507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa243-c56f-4bbd-8705-7efc48178826 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:12.145963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f6fed-a9dc-49a8-97fe-9572f671c575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:12.146730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6fed-a9dc-49a8-97fe-9572f671c575 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:12.147084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6fed-a9dc-49a8-97fe-9572f671c575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:12.147474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6fed-a9dc-49a8-97fe-9572f671c575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:12.148241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f6fed-a9dc-49a8-97fe-9572f671c575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:12.152181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f6fed-a9dc-49a8-97fe-9572f671c575 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:12.165896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1098/2220] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:12.170402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:12.181347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e79f43-442c-4af5-bc25-7fbc35f43139 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:12.187909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:12.192645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:12.192994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:12.193336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] PATH_INFO: `/v3/projects/002000e5606c46c29c87b0fa2fe68b3c/users/5e05c9bdfc114478b5d80f077b970d17/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:12.211737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] RBAC: Authorizing `identity:create_grant(project_id=002000e5606c46c29c87b0fa2fe68b3c, user_id=5e05c9bdfc114478b5d80f077b970d17, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:12.215800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:12.235175 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1123/2221] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:12 2022] PUT /identity/v3/projects/002000e5606c46c29c87b0fa2fe68b3c/users/5e05c9bdfc114478b5d80f077b970d17/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:12.237542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a55fe-fb40-44a4-a53c-d7dedb5b2b5a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:12.237841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a55fe-fb40-44a4-a53c-d7dedb5b2b5a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:12.238126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a55fe-fb40-44a4-a53c-d7dedb5b2b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:12.273288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7d6a55fe-fb40-44a4-a53c-d7dedb5b2b5a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:12.274517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7d6a55fe-fb40-44a4-a53c-d7dedb5b2b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:12.334123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1099/2222] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:12.344174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:12.345883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:12.401112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a45bdd-5cb4-454b-a0b4-67f80aaf376e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:12.404735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:12.405344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:12.405622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:12.405895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:12.414652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:12.416058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:12.426030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1124/2223] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:13.083774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791fc9-78f7-4835-9262-119c743879cc None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:13.084324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791fc9-78f7-4835-9262-119c743879cc None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:13.084324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791fc9-78f7-4835-9262-119c743879cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:13.118214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2f791fc9-78f7-4835-9262-119c743879cc None None] MFA Rules not processed for user `5e05c9bdfc114478b5d80f077b970d17`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:13.119436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2f791fc9-78f7-4835-9262-119c743879cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:13.203256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1100/2224] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:13.213887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:13.215608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:13.296988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ea450-97bb-48da-956c-7cf6c34cc18b service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:13.301699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:13.303206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:13.303850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:13.304284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:13.314829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:13.316672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:13.328048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1125/2225] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:14.890649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7fc1-8048-46e2-a976-a39ed405ea86 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:14.891235 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7fc1-8048-46e2-a976-a39ed405ea86 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:14.891373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7fc1-8048-46e2-a976-a39ed405ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:14.912477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-614c7fc1-8048-46e2-a976-a39ed405ea86 None None] MFA Rules not processed for user `5e05c9bdfc114478b5d80f077b970d17`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:14.913987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-614c7fc1-8048-46e2-a976-a39ed405ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:14.921971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1101/2226] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:14.986760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:14.988173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:15.002879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee242cca-3465-486e-a1f5-af9a3fd276a1 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:15.007775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:15.008489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:15.008821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:15.009093 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] PATH_INFO: `/v3/users/95444302d9384587a921c66a8d900edf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:15.010478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] RBAC: Authorizing `identity:delete_user(user_id=95444302d9384587a921c66a8d900edf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:15.012785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:15.075784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:15.078321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95444302d9384587a921c66a8d900edf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.091159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95444302d9384587a921c66a8d900edf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.114832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95444302d9384587a921c66a8d900edf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.125574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95444302d9384587a921c66a8d900edf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.126339 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95444302d9384587a921c66a8d900edf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.135170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1126/2227] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:14 2022] DELETE /identity/v3/users/95444302d9384587a921c66a8d900edf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:15.140002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:15.140424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:15.140881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:15.167362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:15.168764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:15.173781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:15.249793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1102/2228] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:28:15.260065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:15.261521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:15.327699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09350-8d47-4748-bd3b-be5c737e96da admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:15.331730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b464-adfc-4eb3-8862-74b8ef4cc3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:15.332508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b464-adfc-4eb3-8862-74b8ef4cc3dd service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:15.333156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b464-adfc-4eb3-8862-74b8ef4cc3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:15.333510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b464-adfc-4eb3-8862-74b8ef4cc3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:15.343990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7b464-adfc-4eb3-8862-74b8ef4cc3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:15.346489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7b464-adfc-4eb3-8862-74b8ef4cc3dd service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:15.356898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1127/2229] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:15.709480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:15.710898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:15.738955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26fac-e9ec-4186-9b63-5bcce38fdc69 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:15.743973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:15.744715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:15.744928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:15.745209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] PATH_INFO: `/v3/projects/0c437939c9f048cca5eaac0d883c4780` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:15.746493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c437939c9f048cca5eaac0d883c4780)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:15.748525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:15.811378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c437939c9f048cca5eaac0d883c4780', 'request_id': 'req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.820407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c437939c9f048cca5eaac0d883c4780', 'request_id': 'req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.832037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c437939c9f048cca5eaac0d883c4780', 'request_id': 'req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.848871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] The token cache is being invalidate because project 0c437939c9f048cca5eaac0d883c4780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:15.849866 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.850103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-29a4cde3-0ffc-4fd9-b08b-f8179b725e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:15.851802 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1103/2230] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:15 2022] DELETE /identity/v3/projects/0c437939c9f048cca5eaac0d883c4780 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:15.857279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6381b-e662-44b7-83b8-66666da0aebf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:15.857625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6381b-e662-44b7-83b8-66666da0aebf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:15.857903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6381b-e662-44b7-83b8-66666da0aebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:15.878853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e5e6381b-e662-44b7-83b8-66666da0aebf None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:15.880596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e5e6381b-e662-44b7-83b8-66666da0aebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:15.956656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1128/2231] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:15.964315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07af8d-4ea9-4159-b518-1ee191f8966e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:15.964812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07af8d-4ea9-4159-b518-1ee191f8966e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:15.964926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07af8d-4ea9-4159-b518-1ee191f8966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:15.991827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2a07af8d-4ea9-4159-b518-1ee191f8966e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:15.993205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2a07af8d-4ea9-4159-b518-1ee191f8966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:16.006806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1104/2232] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:16.010947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6381b-e662-44b7-83b8-66666da0aebf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.023356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6381b-e662-44b7-83b8-66666da0aebf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.028746 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.030095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.030539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.030926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.031748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:28:16.032148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.033927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.047477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1129/2233] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 06:28:16.051800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07af8d-4ea9-4159-b518-1ee191f8966e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.062208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07af8d-4ea9-4159-b518-1ee191f8966e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.067613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.068479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.069402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.069903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.070641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.072811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.109378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1105/2234] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:16 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:16.111016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.121819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a342c-3076-415c-92cd-d98102e5b195 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.175674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:16.189276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.190157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.190475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.190726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.191401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.193648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.240247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1130/2235] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:16 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:28:16.241920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.257127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff5b7-ed5c-4b7f-88fc-59cd23caf070 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.263135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.263896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.264252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.264678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.265446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.267150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.281659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1106/2236] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:28:16.283301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.295403 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05952346-5605-4a54-9a91-efa7dd911955 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.300754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f34b5-3bbe-46ef-8f97-9f34820a1933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.301388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f34b5-3bbe-46ef-8f97-9f34820a1933 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.301671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f34b5-3bbe-46ef-8f97-9f34820a1933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.301948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f34b5-3bbe-46ef-8f97-9f34820a1933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.302649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f34b5-3bbe-46ef-8f97-9f34820a1933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.304278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f34b5-3bbe-46ef-8f97-9f34820a1933 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.316803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1131/2237] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:16.321059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.331494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5ecf6-5974-4ae1-afeb-86c329f88bb4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.336478 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.337182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.337640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.337926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] PATH_INFO: `/v3/projects/c3127376a00e4d169ac7b682be88dc3d/users/bf62993ba2814c8b92956e3a0e094036/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.353872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3127376a00e4d169ac7b682be88dc3d, user_id=bf62993ba2814c8b92956e3a0e094036, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.356651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.372111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1107/2238] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:16 2022] PUT /identity/v3/projects/c3127376a00e4d169ac7b682be88dc3d/users/bf62993ba2814c8b92956e3a0e094036/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:16.383658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee500c-4a1e-474b-ab38-136c5e6880f1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.384055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee500c-4a1e-474b-ab38-136c5e6880f1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.384411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee500c-4a1e-474b-ab38-136c5e6880f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.413111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c4ee500c-4a1e-474b-ab38-136c5e6880f1 None None] MFA Rules not processed for user `bf62993ba2814c8b92956e3a0e094036`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:16.414531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c4ee500c-4a1e-474b-ab38-136c5e6880f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:16.493276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1132/2239] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:16.499027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.537275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:16.578391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eea88-420a-41a3-96f2-c5248e0a797f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.583748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.584479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.584906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.585252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.585996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.588162 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.626621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1108/2240] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:16 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:16.630531 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee500c-4a1e-474b-ab38-136c5e6880f1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.643002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee500c-4a1e-474b-ab38-136c5e6880f1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.776268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.776991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.777639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.777934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.778642 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.783610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.824447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1133/2241] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:16 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:16.829060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.858007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67054949-8705-4554-943a-caa3a3c05fbc admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.871647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.872503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.875231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.875762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.876647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.878419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.910466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1109/2242] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:16.911735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:16.924899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c862ca-9eb1-4323-9353-69a74e61e618 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:16.930070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:16.930782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:16.931064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:16.931591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] PATH_INFO: `/v3/projects/a49ba76270404059a3026f0234980d0b/users/5b3c23975b7442c988fce79bdd7ebf25/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:16.954433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49ba76270404059a3026f0234980d0b, user_id=5b3c23975b7442c988fce79bdd7ebf25, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:16.958250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:16.978951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1134/2243] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:16 2022] PUT /identity/v3/projects/a49ba76270404059a3026f0234980d0b/users/5b3c23975b7442c988fce79bdd7ebf25/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:16.980230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:17.002789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754f366-5523-470f-ba17-33d017dd8dd8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:17.084092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cd907-edcd-4561-8d0f-e3dc55716a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:17.084947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cd907-edcd-4561-8d0f-e3dc55716a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:17.085343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cd907-edcd-4561-8d0f-e3dc55716a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:17.085731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cd907-edcd-4561-8d0f-e3dc55716a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:17.086583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cd907-edcd-4561-8d0f-e3dc55716a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:17.088579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cd907-edcd-4561-8d0f-e3dc55716a02 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:17.100324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1110/2244] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:28:17.105182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:17.128007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44141f0-e84a-4950-8841-be104137ba73 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:17.132842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:17.133485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:17.133775 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:17.134655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] PATH_INFO: `/v3/projects/a49ba76270404059a3026f0234980d0b/users/5b3c23975b7442c988fce79bdd7ebf25/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:17.136912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49ba76270404059a3026f0234980d0b, user_id=5b3c23975b7442c988fce79bdd7ebf25, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:17.139849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:17.160252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1135/2245] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:17 2022] PUT /identity/v3/projects/a49ba76270404059a3026f0234980d0b/users/5b3c23975b7442c988fce79bdd7ebf25/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:17.163694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5fa2b-5882-467a-a01a-102a6f15c95d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:17.164064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5fa2b-5882-467a-a01a-102a6f15c95d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:17.165035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5fa2b-5882-467a-a01a-102a6f15c95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:17.248924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8bd5fa2b-5882-467a-a01a-102a6f15c95d None None] MFA Rules not processed for user `5b3c23975b7442c988fce79bdd7ebf25`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:17.250320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8bd5fa2b-5882-467a-a01a-102a6f15c95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:17.356298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1111/2246] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:17.371096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:17.373449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:17.403377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:17.468722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310595b-82dc-47f9-9345-f6ab1e94dbd4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:17.472988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d156e-162e-499a-bf5f-4587ab63dd27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:17.474112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d156e-162e-499a-bf5f-4587ab63dd27 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:17.474544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d156e-162e-499a-bf5f-4587ab63dd27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:17.475391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d156e-162e-499a-bf5f-4587ab63dd27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:17.557072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d156e-162e-499a-bf5f-4587ab63dd27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:17.558707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d156e-162e-499a-bf5f-4587ab63dd27 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:17.571187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1136/2247] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:20.356573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5fa2b-5882-467a-a01a-102a6f15c95d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:20.395880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:20.448621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5fa2b-5882-467a-a01a-102a6f15c95d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:20.456471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:20.457351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:20.458212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:20.458486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] PATH_INFO: `/v3/users/bf62993ba2814c8b92956e3a0e094036` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:20.459705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf62993ba2814c8b92956e3a0e094036)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:20.461701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:20.538264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf62993ba2814c8b92956e3a0e094036'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:20.539315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf62993ba2814c8b92956e3a0e094036'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:20.553236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf62993ba2814c8b92956e3a0e094036'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:20.562839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf62993ba2814c8b92956e3a0e094036'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:20.563830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf62993ba2814c8b92956e3a0e094036'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:20.579426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1112/2248] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:20 2022] DELETE /identity/v3/users/bf62993ba2814c8b92956e3a0e094036 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:20.583966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:20.584508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:20.584508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:20.605006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:20.606338 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:20.676149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1137/2249] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:20.940986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:20.942377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:21.124724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb46a51-a4b9-4910-891b-56a982e6337f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:21.136599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:21.137800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:21.140907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:21.141429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:21.167591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:21.173246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:21.186245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1113/2250] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:21.858472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:21.861986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:21.890723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b80c9-1c9e-4ca1-8ebd-3145dbc9ced4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:21.897044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:21.898530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:21.899009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:21.899484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] PATH_INFO: `/v3/projects/c3127376a00e4d169ac7b682be88dc3d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:21.901228 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3127376a00e4d169ac7b682be88dc3d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:21.903606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:21.962062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3127376a00e4d169ac7b682be88dc3d', 'request_id': 'req-0ef5647f-555e-4fce-9f04-dc236d749a2b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:21.969414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3127376a00e4d169ac7b682be88dc3d', 'request_id': 'req-0ef5647f-555e-4fce-9f04-dc236d749a2b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:21.976019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3127376a00e4d169ac7b682be88dc3d', 'request_id': 'req-0ef5647f-555e-4fce-9f04-dc236d749a2b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:21.978351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:21.989421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] The token cache is being invalidate because project c3127376a00e4d169ac7b682be88dc3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:21.989695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:21.990344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:21.992224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1138/2251] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:21 2022] DELETE /identity/v3/projects/c3127376a00e4d169ac7b682be88dc3d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:21.996107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:22.008593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4961-55b7-459a-8ab8-2a24afdc65fc service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:22.060924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:22.083652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:22.084352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:22.084679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:22.084964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] PATH_INFO: `/v3/users/5b3c23975b7442c988fce79bdd7ebf25` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:22.087129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3c23975b7442c988fce79bdd7ebf25)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:22.089792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:22.163046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3c23975b7442c988fce79bdd7ebf25'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.166931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3c23975b7442c988fce79bdd7ebf25'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.188957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3c23975b7442c988fce79bdd7ebf25'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.201757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3c23975b7442c988fce79bdd7ebf25'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.202689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-37dcb30a-a670-4b47-b309-0e099a71c127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3c23975b7442c988fce79bdd7ebf25'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.229936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1114/2252] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:21 2022] DELETE /identity/v3/users/5b3c23975b7442c988fce79bdd7ebf25 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 06:28:22.522841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:22.524364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:22.555109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5647f-555e-4fce-9f04-dc236d749a2b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:22.615242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:22.616087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:22.616385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:22.617182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] PATH_INFO: `/v3/projects/a49ba76270404059a3026f0234980d0b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:22.618529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49ba76270404059a3026f0234980d0b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:22.620724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:22.708766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49ba76270404059a3026f0234980d0b', 'request_id': 'req-852cb315-4fd1-42ac-a353-86c930b2b1d2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.739421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49ba76270404059a3026f0234980d0b', 'request_id': 'req-852cb315-4fd1-42ac-a353-86c930b2b1d2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.749530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49ba76270404059a3026f0234980d0b', 'request_id': 'req-852cb315-4fd1-42ac-a353-86c930b2b1d2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.762454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] The token cache is being invalidate because project a49ba76270404059a3026f0234980d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:22.762758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.763822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-852cb315-4fd1-42ac-a353-86c930b2b1d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:22.766174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1139/2253] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:22 2022] DELETE /identity/v3/projects/a49ba76270404059a3026f0234980d0b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:22.770727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3952-3673-44ed-a8e5-218ea5b20d8d None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:22.771482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3952-3673-44ed-a8e5-218ea5b20d8d None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:22.771482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3952-3673-44ed-a8e5-218ea5b20d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:22.794885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-deda3952-3673-44ed-a8e5-218ea5b20d8d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:22.795816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-deda3952-3673-44ed-a8e5-218ea5b20d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:22.864059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1115/2254] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:22.872173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fe70a-1a60-44d4-abfa-891c2b52d638 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:22.873284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fe70a-1a60-44d4-abfa-891c2b52d638 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:22.873455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fe70a-1a60-44d4-abfa-891c2b52d638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:22.893257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c33fe70a-1a60-44d4-abfa-891c2b52d638 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:22.894459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c33fe70a-1a60-44d4-abfa-891c2b52d638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:22.902334 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1140/2255] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:22.906709 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda3952-3673-44ed-a8e5-218ea5b20d8d None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:22.916698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda3952-3673-44ed-a8e5-218ea5b20d8d None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:22.921871 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:22.922482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:22.922765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:22.923041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:22.923730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:28:22.924038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:22.925676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:22.938212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1116/2256] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:28:22.942392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fe70a-1a60-44d4-abfa-891c2b52d638 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:22.951943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fe70a-1a60-44d4-abfa-891c2b52d638 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:22.956575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:22.957184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:22.957468 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:22.958002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:22.958677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:22.960555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:22.987222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1141/2257] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:22 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:22.991403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:23.002179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0e60-a0e0-4974-8161-dd228a5bfcef admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:23.049925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:23.061286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:23.061921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:23.062379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:23.062715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:23.063574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:23.065529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:23.087058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1117/2258] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:22 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:23.091130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:23.103140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb48c5f-1320-4755-9b1e-a62b9fa99bd0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:23.109058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:23.109597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:23.109904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:23.110253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:23.111058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:23.112759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:23.124285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1142/2259] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:23.128418 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:23.140797 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9972ee-6c93-454b-8c6e-acdce88cfa4d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:23.145792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13816a89-0252-4bb1-9c9f-a36f3c27757b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:23.146452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13816a89-0252-4bb1-9c9f-a36f3c27757b admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:23.146740 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13816a89-0252-4bb1-9c9f-a36f3c27757b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:23.147018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13816a89-0252-4bb1-9c9f-a36f3c27757b admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:23.147713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13816a89-0252-4bb1-9c9f-a36f3c27757b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:23.149361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13816a89-0252-4bb1-9c9f-a36f3c27757b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:23.161007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1118/2260] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:23.170324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:23.189743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70b13-1184-4943-ad97-ca846f0d0e89 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:23.194910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:23.195596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:23.195885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:23.196165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] PATH_INFO: `/v3/projects/f106f02a007148939a6e820854d475e4/users/edcd7d62f18c44b69030a0a72a811b4e/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:23.213391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f106f02a007148939a6e820854d475e4, user_id=edcd7d62f18c44b69030a0a72a811b4e, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:23.216087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:23.238311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1143/2261] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:23 2022] PUT /identity/v3/projects/f106f02a007148939a6e820854d475e4/users/edcd7d62f18c44b69030a0a72a811b4e/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:23.321988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47660590-f6d3-457f-aec2-da368288804e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:23.322280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47660590-f6d3-457f-aec2-da368288804e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:23.322606 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47660590-f6d3-457f-aec2-da368288804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:23.353649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-47660590-f6d3-457f-aec2-da368288804e None None] MFA Rules not processed for user `edcd7d62f18c44b69030a0a72a811b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:23.355410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-47660590-f6d3-457f-aec2-da368288804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:23.447433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1119/2262] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:23.469869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:23.471658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:23.509274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:23.557458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d5f43-d252-4f4c-b0f5-b90a78ce24e2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:23.564598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:23.565547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:23.565834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:23.566321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:23.576491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:23.577941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:23.587241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1144/2263] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:33.445117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:33.583104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=18 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:34.615560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e485b4-0236-48f2-bc0f-062354cd76ea None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:34.615985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e485b4-0236-48f2-bc0f-062354cd76ea None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:34.616145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e485b4-0236-48f2-bc0f-062354cd76ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:34.636077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-56e485b4-0236-48f2-bc0f-062354cd76ea None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:34.637298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-56e485b4-0236-48f2-bc0f-062354cd76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:34.693122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1120/2264] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:28:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:34.736604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:34.738025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:34.810813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4da727-37b2-4612-9a3d-7efd80d05848 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:34.817296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a23c95-7d90-4a71-afe9-67a20f5d2232 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:34.818035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23c95-7d90-4a71-afe9-67a20f5d2232 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:34.818313 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23c95-7d90-4a71-afe9-67a20f5d2232 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:34.818593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23c95-7d90-4a71-afe9-67a20f5d2232 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:34.828742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a23c95-7d90-4a71-afe9-67a20f5d2232 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:34.830393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a23c95-7d90-4a71-afe9-67a20f5d2232 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:34.839185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1145/2265] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:28:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:28:44.686664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=25 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:44.835777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=32 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:47.273778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e485b4-0236-48f2-bc0f-062354cd76ea None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:47.345640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e485b4-0236-48f2-bc0f-062354cd76ea None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:47.350945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:47.351631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:47.351916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:47.352210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] PATH_INFO: `/v3/users/edcd7d62f18c44b69030a0a72a811b4e` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:47.353489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcd7d62f18c44b69030a0a72a811b4e)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:47.355727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:47.412471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcd7d62f18c44b69030a0a72a811b4e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:47.413504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcd7d62f18c44b69030a0a72a811b4e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:47.428570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcd7d62f18c44b69030a0a72a811b4e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:47.437048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcd7d62f18c44b69030a0a72a811b4e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:47.438028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcd7d62f18c44b69030a0a72a811b4e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:47.456845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1121/2266] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:28:47 2022] DELETE /identity/v3/users/edcd7d62f18c44b69030a0a72a811b4e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:47.461729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:47.462285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:47.462514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:47.503386 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:47.504684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:47.571480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1146/2267] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:47.581655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:47.583381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:47.653081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c9449-efe3-4030-a890-61329e965b1a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:47.658717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a180d52-d491-40df-8d25-0bf7e3774b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:47.659415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180d52-d491-40df-8d25-0bf7e3774b1c service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:47.659853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180d52-d491-40df-8d25-0bf7e3774b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:47.660461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180d52-d491-40df-8d25-0bf7e3774b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:47.673399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a180d52-d491-40df-8d25-0bf7e3774b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:47.675668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a180d52-d491-40df-8d25-0bf7e3774b1c service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:47.688830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1122/2268] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:48.173423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:48.175598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:48.230994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5ea45-15a5-44c5-86e7-8721f30cf78c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:48.267415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:48.271239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.288320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.290082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] PATH_INFO: `/v3/projects/f106f02a007148939a6e820854d475e4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.293470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f106f02a007148939a6e820854d475e4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:48.295839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:48.433065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f106f02a007148939a6e820854d475e4', 'request_id': 'req-a5134838-be0b-45cd-836e-8f44af8394d2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:48.445974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f106f02a007148939a6e820854d475e4', 'request_id': 'req-a5134838-be0b-45cd-836e-8f44af8394d2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:48.454290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f106f02a007148939a6e820854d475e4', 'request_id': 'req-a5134838-be0b-45cd-836e-8f44af8394d2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:48.470173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] The token cache is being invalidate because project f106f02a007148939a6e820854d475e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:28:48.471010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:48.471140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a5134838-be0b-45cd-836e-8f44af8394d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:28:48.472822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1147/2269] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:28:48 2022] DELETE /identity/v3/projects/f106f02a007148939a6e820854d475e4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:48.478059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327c331-2d2a-4eec-8f24-dd3341f12313 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.478681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327c331-2d2a-4eec-8f24-dd3341f12313 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.478681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327c331-2d2a-4eec-8f24-dd3341f12313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.503814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b327c331-2d2a-4eec-8f24-dd3341f12313 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:48.505154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b327c331-2d2a-4eec-8f24-dd3341f12313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:48.524348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:48.577060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1123/2270] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:48.587066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62084260-1c61-4f5a-937b-bb36bd15ee0d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.587066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62084260-1c61-4f5a-937b-bb36bd15ee0d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.587066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62084260-1c61-4f5a-937b-bb36bd15ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.629069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-62084260-1c61-4f5a-937b-bb36bd15ee0d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:48.630564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-62084260-1c61-4f5a-937b-bb36bd15ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:48.654190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1148/2271] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:48.658662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327c331-2d2a-4eec-8f24-dd3341f12313 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:48.670494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327c331-2d2a-4eec-8f24-dd3341f12313 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:48.676288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:48.677475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.678068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.678557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.679466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:28:48.679999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:48.681868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:48.696229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1124/2272] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:28:48.701207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62084260-1c61-4f5a-937b-bb36bd15ee0d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:48.715318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62084260-1c61-4f5a-937b-bb36bd15ee0d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:48.738045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:48.738698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.738986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.739327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.740693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:48.742895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:48.784506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1149/2273] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:28:48 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:28:48.792986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:48.807006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8ed0-01b5-4e0e-970d-0bd62facb8d3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:48.880085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:48.880965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.881580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.881987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.882592 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:48.884704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:48.915368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1125/2274] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:28:48 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:28:48.920109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:48.933042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61a3a4-3d25-4e6a-9cf7-3f7708dcbb31 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:48.940407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:48.941557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.942173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.942736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.944032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:48.947354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:48.967766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1150/2275] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:28:48.971919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:48.984626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7945782-5354-47a2-ba91-ffe08ac8828d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:48.990201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b7c88-b5e2-45a1-bd85-b2f625ff5bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:48.990904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7c88-b5e2-45a1-bd85-b2f625ff5bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:48.991425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7c88-b5e2-45a1-bd85-b2f625ff5bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:48.991799 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7c88-b5e2-45a1-bd85-b2f625ff5bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:48.992594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b7c88-b5e2-45a1-bd85-b2f625ff5bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:48.994328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b7c88-b5e2-45a1-bd85-b2f625ff5bfb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:49.008585 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1126/2276] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:28:49.013045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:49.023929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26853e2-1ac1-4a77-9cb8-8d85378f2d8d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:49.029458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:49.030324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:49.030772 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:49.031219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] PATH_INFO: `/v3/projects/6b5916906d6048bb96c13c457bf367e3/users/1d436bd35f48454da2dc2e607247fe02/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:49.047385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5916906d6048bb96c13c457bf367e3, user_id=1d436bd35f48454da2dc2e607247fe02, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:49.050922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:49.068901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1151/2277] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:28:49 2022] PUT /identity/v3/projects/6b5916906d6048bb96c13c457bf367e3/users/1d436bd35f48454da2dc2e607247fe02/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:28:49.074308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2007359-4474-4de2-b0a8-79f554dbee6c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:49.075400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2007359-4474-4de2-b0a8-79f554dbee6c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:49.075633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2007359-4474-4de2-b0a8-79f554dbee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:49.095862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c2007359-4474-4de2-b0a8-79f554dbee6c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:49.097361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c2007359-4474-4de2-b0a8-79f554dbee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:49.181213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1127/2278] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:49.188627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:49.190559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:49.223025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:49.269003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eed08-9732-4f75-aa2a-159c3f58bbd9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:49.273164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66b143-eeb5-456e-9108-47662920049b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:49.274066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66b143-eeb5-456e-9108-47662920049b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:49.274357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66b143-eeb5-456e-9108-47662920049b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:49.274639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66b143-eeb5-456e-9108-47662920049b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:49.286665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66b143-eeb5-456e-9108-47662920049b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:49.288751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66b143-eeb5-456e-9108-47662920049b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:49.301078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1152/2279] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:28:54.796124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:54.797781 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:54.798485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:54.884865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] MFA Rules not processed for user `b34e3e2e2e6049428295588959ca595c`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:54.886155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:54.968388 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:28:55.065241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1128/2280] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:28:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:55.156037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a57a-5ad6-436e-9445-6f01fe51ae82 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:55.156919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a57a-5ad6-436e-9445-6f01fe51ae82 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:55.157381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a57a-5ad6-436e-9445-6f01fe51ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:55.198737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ea60a57a-5ad6-436e-9445-6f01fe51ae82 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:55.202054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ea60a57a-5ad6-436e-9445-6f01fe51ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:55.350426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1153/2281] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:28:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:55.364527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:55.366339 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:55.536705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50d9ae-1e8d-4e2a-b842-378207dec8de None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:55.553551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:55.554211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:55.554511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:55.554780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:55.575893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:55.578782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:55.610553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1129/2282] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:28:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:28:57.319512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68620bd-18ac-41d0-ac97-9bbefc8a0722 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:57.320045 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68620bd-18ac-41d0-ac97-9bbefc8a0722 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:57.320154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68620bd-18ac-41d0-ac97-9bbefc8a0722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:57.358562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a68620bd-18ac-41d0-ac97-9bbefc8a0722 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:28:57.362802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a68620bd-18ac-41d0-ac97-9bbefc8a0722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:28:57.384722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1154/2283] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:28:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:28:57.403326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:28:57.422615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539088f-5e93-4bc3-abb6-9b28bb980263 service swift] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:28:57.444334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:28:57.445570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:28:57.446175 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:28:57.446611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:28:57.457130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:28:57.463362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:28:57.484425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1130/2284] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:28:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:29:00.292126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745723-5dc8-44f4-8c5a-92cd42cc6d30 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:00.292748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745723-5dc8-44f4-8c5a-92cd42cc6d30 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:00.292748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745723-5dc8-44f4-8c5a-92cd42cc6d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:00.347020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-49745723-5dc8-44f4-8c5a-92cd42cc6d30 None None] MFA Rules not processed for user `1d436bd35f48454da2dc2e607247fe02`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:00.348376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-49745723-5dc8-44f4-8c5a-92cd42cc6d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:00.510373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:00.523689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcb918-97a4-4f77-a3d8-ff97083ce7ea service swift] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:00.526298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1155/2285] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:00.533040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bb3b4-67d4-4343-a6f1-4f7f8114aad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:00.534039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb3b4-67d4-4343-a6f1-4f7f8114aad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:00.534558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb3b4-67d4-4343-a6f1-4f7f8114aad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:00.535033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb3b4-67d4-4343-a6f1-4f7f8114aad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:00.536728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bb3b4-67d4-4343-a6f1-4f7f8114aad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:00.539526 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bb3b4-67d4-4343-a6f1-4f7f8114aad7 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:00.646296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745723-5dc8-44f4-8c5a-92cd42cc6d30 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:00.658764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1131/2286] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:29:00 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:29:00.663107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745723-5dc8-44f4-8c5a-92cd42cc6d30 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:00.789408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:00.813953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:00.815666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:00.817579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:00.817947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:00.818725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:00.835354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:00.866949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882c5a0-fd20-4611-a188-92f7b0aef796 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:00.868030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882c5a0-fd20-4611-a188-92f7b0aef796 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:00.868428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882c5a0-fd20-4611-a188-92f7b0aef796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:00.936301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1156/2287] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:29:00 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:29:00.936520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5882c5a0-fd20-4611-a188-92f7b0aef796 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:00.936520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5882c5a0-fd20-4611-a188-92f7b0aef796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:00.937749 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:00.958194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a479cdc-e3ed-4d0a-ba86-b85739a8ac27 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:00.973681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:00.974477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:00.976855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:00.977624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:00.978076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:00.984198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:01.015449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1157/2288] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:29:01.016702 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:01.056768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5a73c-b306-45b8-8050-5d198f9a68da admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:01.060982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1132/2289] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 29 06:29:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:29:01.080428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:01.081544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:01.082064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:01.083044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] PATH_INFO: `/v3/projects/2f8be70807e2436e840ac66dca5a11d0/users/648d19509ebe4616a24bd832fb762872/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:01.090199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882c5a0-fd20-4611-a188-92f7b0aef796 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:01.106168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8be70807e2436e840ac66dca5a11d0, user_id=648d19509ebe4616a24bd832fb762872, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:01.109966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:01.137278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:01.140582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1158/2290] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:29:01 2022] PUT /identity/v3/projects/2f8be70807e2436e840ac66dca5a11d0/users/648d19509ebe4616a24bd832fb762872/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:01.145435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:01.177504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882c5a0-fd20-4611-a188-92f7b0aef796 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:01.261123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df42e99-1616-4ba0-89df-5cf4742ef684 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:01.262267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42e99-1616-4ba0-89df-5cf4742ef684 service swift] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:01.262548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42e99-1616-4ba0-89df-5cf4742ef684 service swift] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:01.262820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42e99-1616-4ba0-89df-5cf4742ef684 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:01.272340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a657c1-24ec-402e-a3dd-a6c5bdf62c25 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:01.285520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:01.286165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:01.286449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:01.286725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:01.287417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:01.291431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:01.318535 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1159/2291] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:29:01.320018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:01.347447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392dc39-3c20-4a44-a329-40d1caff38ba admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:01.361534 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:01.362089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:01.362890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:01.363171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] PATH_INFO: `/v3/projects/2f8be70807e2436e840ac66dca5a11d0/users/648d19509ebe4616a24bd832fb762872/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:01.366479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df42e99-1616-4ba0-89df-5cf4742ef684 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:01.368307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df42e99-1616-4ba0-89df-5cf4742ef684 service swift] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:01.368993 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8be70807e2436e840ac66dca5a11d0, user_id=648d19509ebe4616a24bd832fb762872, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:01.371740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:01.388215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1133/2292] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:29:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:29:01.394598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1160/2293] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:29:01 2022] PUT /identity/v3/projects/2f8be70807e2436e840ac66dca5a11d0/users/648d19509ebe4616a24bd832fb762872/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:01.993301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ceeb3-875a-49a0-9287-483eec95875e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:01.994225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ceeb3-875a-49a0-9287-483eec95875e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:01.994709 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ceeb3-875a-49a0-9287-483eec95875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:02.028654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-ac0ceeb3-875a-49a0-9287-483eec95875e None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:02.030358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ac0ceeb3-875a-49a0-9287-483eec95875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:02.227741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1134/2294] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 29 06:29:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:02.233356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:02.388134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39d17a-2df7-4303-814b-73336765b735 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:02.401335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:02.402710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:02.403129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:02.403415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:02.429567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:02.431383 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:02.471006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1161/2295] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:29:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:29:03.755929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e342c-3b0b-472f-8dd2-a57e0908eb52 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:03.756429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e342c-3b0b-472f-8dd2-a57e0908eb52 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:03.756521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e342c-3b0b-472f-8dd2-a57e0908eb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:03.791427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d81e342c-3b0b-472f-8dd2-a57e0908eb52 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:03.793375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d81e342c-3b0b-472f-8dd2-a57e0908eb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:03.817830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1135/2296] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:29:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:03.829578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:03.845552 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9ba87-06d8-46a4-88d7-ec5fe4ffaa2e service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:03.865325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:03.865959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:03.866254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:03.866541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:03.885811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:03.888854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:03.918632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1162/2297] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:29:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:29:04.811209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:04.812326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:04.812843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:04.872182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:04.874354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:04.893735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1136/2298] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 29 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:04.901456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:04.912951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:04.925700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1737f-861b-4f5e-b51a-8f7a40c35302 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:04.940558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:04.941314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:04.942013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:04.942470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:04.970136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:04.973951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:05.016075 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1163/2299] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:29:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:29:07.584101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:07.586579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:07.605876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:07.696358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aedec-9da7-4225-ba6b-c221bbdc1028 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:07.707116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:07.708978 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:07.709573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:07.710352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] PATH_INFO: `/v3/users/55bb264bd9834f5fa61b58362ec48d31` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:07.712708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=55bb264bd9834f5fa61b58362ec48d31)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:07.715732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:07.833506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55bb264bd9834f5fa61b58362ec48d31'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:07.835262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55bb264bd9834f5fa61b58362ec48d31'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:07.879062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55bb264bd9834f5fa61b58362ec48d31'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:07.889426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55bb264bd9834f5fa61b58362ec48d31'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:07.890541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e0f53f5b-0123-42fb-aa08-da843646c85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55bb264bd9834f5fa61b58362ec48d31'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:07.921520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1137/2300] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:29:07 2022] DELETE /identity/v3/users/55bb264bd9834f5fa61b58362ec48d31 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:08.133887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:08.137786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:08.159118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd9e33-8b29-4a27-b752-c139140e9dae service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:08.241811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:08.243212 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:08.243742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:08.244230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] PATH_INFO: `/v3/users/e8d0f88d70ba4646a10c89d9d32de574` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:08.246798 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d0f88d70ba4646a10c89d9d32de574)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:08.249036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:08.331197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d0f88d70ba4646a10c89d9d32de574'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.341276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d0f88d70ba4646a10c89d9d32de574'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.359625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d0f88d70ba4646a10c89d9d32de574'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.376749 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d0f88d70ba4646a10c89d9d32de574'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.378145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d0f88d70ba4646a10c89d9d32de574'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.398445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1164/2301] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:29:08 2022] DELETE /identity/v3/users/e8d0f88d70ba4646a10c89d9d32de574 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:08.400457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:08.400854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:08.401123 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:08.452009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:08.453246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:08.525433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:08.528476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:08.574721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1138/2302] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:08.585442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:08.587466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:08.628886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:08.699629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe32ee0-04d8-4181-8299-16f4ed355274 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:08.708233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b849ed4-4342-4887-9ffe-7c70a2447401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:08.709154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b849ed4-4342-4887-9ffe-7c70a2447401 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:08.709632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b849ed4-4342-4887-9ffe-7c70a2447401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:08.710008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b849ed4-4342-4887-9ffe-7c70a2447401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:08.746405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b849ed4-4342-4887-9ffe-7c70a2447401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:08.748268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b849ed4-4342-4887-9ffe-7c70a2447401 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:08.768708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235364e2-feb3-4548-a1e0-b63474bd10e6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:08.778360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:08.779067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:08.779354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:08.779634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] PATH_INFO: `/v3/projects/c241dc0f043e46c8a01503a1c343abf9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:08.785502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c241dc0f043e46c8a01503a1c343abf9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:08.787496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:08.794134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1139/2303] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:29:08.816516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:08.937924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c241dc0f043e46c8a01503a1c343abf9', 'request_id': 'req-da078f9b-63bf-4da6-a10c-98d23b4a041c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.961987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c241dc0f043e46c8a01503a1c343abf9', 'request_id': 'req-da078f9b-63bf-4da6-a10c-98d23b4a041c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.970274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c241dc0f043e46c8a01503a1c343abf9', 'request_id': 'req-da078f9b-63bf-4da6-a10c-98d23b4a041c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.983397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] The token cache is being invalidate because project c241dc0f043e46c8a01503a1c343abf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:29:08.983791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.985022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:08.994726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135400e-1fab-43a6-a374-322fbd436fc8 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:08.994726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135400e-1fab-43a6-a374-322fbd436fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:08.994726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135400e-1fab-43a6-a374-322fbd436fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:08.999487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1165/2304] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:29:08 2022] DELETE /identity/v3/projects/c241dc0f043e46c8a01503a1c343abf9 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:09.020953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3135400e-1fab-43a6-a374-322fbd436fc8 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:09.022294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3135400e-1fab-43a6-a374-322fbd436fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:09.077966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.079828 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:09.138957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1140/2305] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:09.140589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253ff2-a312-4e4f-adcc-ddcf14513895 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.140941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253ff2-a312-4e4f-adcc-ddcf14513895 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.141298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253ff2-a312-4e4f-adcc-ddcf14513895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.165733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-4b253ff2-a312-4e4f-adcc-ddcf14513895 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:09.167116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4b253ff2-a312-4e4f-adcc-ddcf14513895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:09.184228 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1141/2306] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:09.185990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b253ff2-a312-4e4f-adcc-ddcf14513895 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.208905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b253ff2-a312-4e4f-adcc-ddcf14513895 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.215901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.217051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.217173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.217503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.218285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:29:09.218598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.221359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.277756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1142/2307] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:29:09.278853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.281281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da078f9b-63bf-4da6-a10c-98d23b4a041c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.297153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.297846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.298265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.298628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] PATH_INFO: `/v3/projects/77d0443b289548b1b6e96bb268977c69` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.301913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384715b-b026-4f14-8373-1681d9fa13bb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.302810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d0443b289548b1b6e96bb268977c69)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.306086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.325480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.326148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.326434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.326715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.327396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.329957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.384438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d0443b289548b1b6e96bb268977c69', 'request_id': 'req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:09.396206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d0443b289548b1b6e96bb268977c69', 'request_id': 'req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:09.410332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d0443b289548b1b6e96bb268977c69', 'request_id': 'req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:09.411146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1143/2308] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:29:09 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:29:09.412127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.430096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] The token cache is being invalidate because project 77d0443b289548b1b6e96bb268977c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:29:09.430678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:09.431541 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a68ab88e-416f-4cd6-a3d4-967e7a39c2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:09.438751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1166/2309] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:29:09 2022] DELETE /identity/v3/projects/77d0443b289548b1b6e96bb268977c69 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:09.443773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171912c0-1820-4663-b542-2b76f635de7f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.444077 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171912c0-1820-4663-b542-2b76f635de7f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.444371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171912c0-1820-4663-b542-2b76f635de7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.474685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-171912c0-1820-4663-b542-2b76f635de7f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:09.476754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-171912c0-1820-4663-b542-2b76f635de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:09.560024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:09.619133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f14c-d53d-49fe-9533-265993075cee admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.635473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.636125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.636982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.637263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.637939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.640149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.661039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1167/2310] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:09.669497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c5d8e-b9d2-4aa7-9dab-a4adfc4d4f0e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.670614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c5d8e-b9d2-4aa7-9dab-a4adfc4d4f0e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.671875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c5d8e-b9d2-4aa7-9dab-a4adfc4d4f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.696494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1144/2311] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:29:09 2022] POST /identity/v3/users => generated 284 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:29:09.701127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.706795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:09.721000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-de0c5d8e-b9d2-4aa7-9dab-a4adfc4d4f0e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:09.726647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-de0c5d8e-b9d2-4aa7-9dab-a4adfc4d4f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:09.738535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf703b-c23b-41fd-9f25-5c3b9ccf9b7d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.748921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1168/2312] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:09.752300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c5d8e-b9d2-4aa7-9dab-a4adfc4d4f0e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.753774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.755194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.757405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.757895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.763117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.765618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.780520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c5d8e-b9d2-4aa7-9dab-a4adfc4d4f0e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.796395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.798886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.799337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.799767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1145/2313] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:29:09.800223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.801293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:29:09.802362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.803430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.809437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.828487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40585-82fd-4ebe-b3e5-59c7f1b2b36a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.836974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1169/2314] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:29:09.840797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.845294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.849598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.850531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.851141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.853649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.857586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.863266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:09.874051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe099e-e671-4dcc-a79a-6dc30af44f38 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.890129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1146/2315] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:29:09.891116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.891926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.892394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.893417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.893862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.895377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:09.899770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:09.933795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ac835-ebcc-4209-bac0-c31f56f47a55 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:09.950588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feebda8-ac5b-4c2e-a76c-a19f393c7704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:09.951290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feebda8-ac5b-4c2e-a76c-a19f393c7704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:09.951576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feebda8-ac5b-4c2e-a76c-a19f393c7704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:09.951860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feebda8-ac5b-4c2e-a76c-a19f393c7704 admin admin] PATH_INFO: `/v3/projects/6e49e72350164b29b0c82de9fd0c714f/users/32968493c6764a09b1f68cd6f4c3acfc/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:09.983616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1170/2316] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:29:09 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:29:09.987918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:09.998250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feebda8-ac5b-4c2e-a76c-a19f393c7704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e49e72350164b29b0c82de9fd0c714f, user_id=32968493c6764a09b1f68cd6f4c3acfc, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:10.002466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feebda8-ac5b-4c2e-a76c-a19f393c7704 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:10.040852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1147/2317] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:29:09 2022] PUT /identity/v3/projects/6e49e72350164b29b0c82de9fd0c714f/users/32968493c6764a09b1f68cd6f4c3acfc/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:10.045082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:10.045370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:10.045683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:10.114273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] MFA Rules not processed for user `32968493c6764a09b1f68cd6f4c3acfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:10.117435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:10.150414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a44b07-615e-4499-9d4b-e70c50c6b130 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:10.221939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1148/2318] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:10.232303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:10.235957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:10.250913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:10.251915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:10.252358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:10.252841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:10.254142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:10.257690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:10.332860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1171/2319] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:29:09 2022] POST /identity/v3/users => generated 296 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 06:29:10.337107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:10.362143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b8a09-55c4-4578-974a-d8ba98361321 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:10.364194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f2682-66b9-4f5e-a89d-2f6ca248e1b9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:10.369382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4b8f7545934151aafa986da871b944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:10.372182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:10.372631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:10.373064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:10.379794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:10.381496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:10.382118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:10.382733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:10.384214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:10.389612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:10.402853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1172/2320] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:29:10.406950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:10.414252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:10.416129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:10.418447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058a4d0-f104-4054-981e-cf2391ed2b42 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:10.439379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f1acc-f383-4d9a-a9e1-0ceb442919e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:10.440029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f1acc-f383-4d9a-a9e1-0ceb442919e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:10.440319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f1acc-f383-4d9a-a9e1-0ceb442919e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:10.440633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f1acc-f383-4d9a-a9e1-0ceb442919e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:10.441343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f1acc-f383-4d9a-a9e1-0ceb442919e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:10.442951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f1acc-f383-4d9a-a9e1-0ceb442919e2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:10.446325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1149/2321] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:29:10.461905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1173/2322] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:29:10.462462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:10.479258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263761c-50e3-4bc8-8df2-7167969568c3 service glance] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:10.485256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:10.486471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:10.487027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:10.487466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] PATH_INFO: `/v3/projects/b6d476aafa4a4ed7964c35cef261021f/users/f1bf0e9a2117423aa3372e025a131e9c/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:10.540001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d476aafa4a4ed7964c35cef261021f, user_id=f1bf0e9a2117423aa3372e025a131e9c, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:10.543638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:10.580686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1150/2323] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:29:10 2022] PUT /identity/v3/projects/b6d476aafa4a4ed7964c35cef261021f/users/f1bf0e9a2117423aa3372e025a131e9c/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:10.581060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e59b-097c-4e30-ab16-9575bebc0cbe None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:10.581519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e59b-097c-4e30-ab16-9575bebc0cbe None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:10.581806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e59b-097c-4e30-ab16-9575bebc0cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:10.600317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:10.676573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3d59e59b-097c-4e30-ab16-9575bebc0cbe None None] MFA Rules not processed for user `f1bf0e9a2117423aa3372e025a131e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:10.676573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d59e59b-097c-4e30-ab16-9575bebc0cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:10.829480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1174/2324] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:10.843437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:10.845891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:10.945916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:10.977751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba97287-091e-4fae-b3cc-32c4d7966d65 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:11.004391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:11.009704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:11.010032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:11.010256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:11.060451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:11.068726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:11.102051 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1151/2325] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:29:12.824376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfbbd7-56e9-43fd-8812-fcdac618b131 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:12.824997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfbbd7-56e9-43fd-8812-fcdac618b131 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:12.824997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfbbd7-56e9-43fd-8812-fcdac618b131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:12.912700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-53bfbbd7-56e9-43fd-8812-fcdac618b131 None None] MFA Rules not processed for user `648d19509ebe4616a24bd832fb762872`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:12.915308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-53bfbbd7-56e9-43fd-8812-fcdac618b131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:13.119181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1175/2326] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:13.128934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:13.131028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:13.348485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1397a-7ec3-4721-99e8-fe146f57a5a3 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:13.355804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa046d-8370-40b9-8311-9de9ed67a832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:13.356763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa046d-8370-40b9-8311-9de9ed67a832 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:13.357157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa046d-8370-40b9-8311-9de9ed67a832 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:13.358344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa046d-8370-40b9-8311-9de9ed67a832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:13.360330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-90fa046d-8370-40b9-8311-9de9ed67a832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:13.451111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa046d-8370-40b9-8311-9de9ed67a832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:13.455021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa046d-8370-40b9-8311-9de9ed67a832 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:13.469402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1152/2327] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:29:15.415838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153369b-9a39-4713-9ca8-ea3168f6a10a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:15.416380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153369b-9a39-4713-9ca8-ea3168f6a10a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:15.418611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153369b-9a39-4713-9ca8-ea3168f6a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:15.447521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a153369b-9a39-4713-9ca8-ea3168f6a10a None None] MFA Rules not processed for user `1d436bd35f48454da2dc2e607247fe02`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:15.450094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a153369b-9a39-4713-9ca8-ea3168f6a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:15.469177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1176/2328] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:23.464268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=63 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:25.436367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=67 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:27.719210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae6cc8-35b1-4cfc-bd07-e4f79f1adb90 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:27.719765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae6cc8-35b1-4cfc-bd07-e4f79f1adb90 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:27.719936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae6cc8-35b1-4cfc-bd07-e4f79f1adb90 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:27.722630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1153/2329] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:29:27.735921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a47d3-e4a9-4973-95bf-2a0a2fca9b4b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:27.736443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a47d3-e4a9-4973-95bf-2a0a2fca9b4b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:27.737036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a47d3-e4a9-4973-95bf-2a0a2fca9b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:27.773982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bd8a47d3-e4a9-4973-95bf-2a0a2fca9b4b None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:27.775390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bd8a47d3-e4a9-4973-95bf-2a0a2fca9b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:27.787328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1177/2330] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:27.801612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae6cc8-35b1-4cfc-bd07-e4f79f1adb90 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:27.886648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae6cc8-35b1-4cfc-bd07-e4f79f1adb90 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:27.891703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f0982-38ac-45bd-96ce-e0a084b17752 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4b8f7545934151aafa986da871b944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:27.892308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0982-38ac-45bd-96ce-e0a084b17752 service glance] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:27.892628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0982-38ac-45bd-96ce-e0a084b17752 service glance] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:27.892906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0982-38ac-45bd-96ce-e0a084b17752 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:27.904728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f0982-38ac-45bd-96ce-e0a084b17752 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:27.906465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f0982-38ac-45bd-96ce-e0a084b17752 service glance] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:27.919067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1154/2331] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:29:28.024380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514bda7-f6e7-4354-8138-9cbe46043c26 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:28.024709 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514bda7-f6e7-4354-8138-9cbe46043c26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:28.024996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514bda7-f6e7-4354-8138-9cbe46043c26 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:28.026976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1178/2332] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:29:28.032208 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b8029-3a3f-4ee6-b8b7-8556504434d3 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:28.032517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b8029-3a3f-4ee6-b8b7-8556504434d3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:28.032806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b8029-3a3f-4ee6-b8b7-8556504434d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:28.053105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-be6b8029-3a3f-4ee6-b8b7-8556504434d3 None None] MFA Rules not processed for user `edf0a00538cf4e10b3bf266f48ea8cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:28.054291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-be6b8029-3a3f-4ee6-b8b7-8556504434d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:28.105428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1155/2333] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:29:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:28.117215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514bda7-f6e7-4354-8138-9cbe46043c26 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:28.118912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9514bda7-f6e7-4354-8138-9cbe46043c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:28.132777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514bda7-f6e7-4354-8138-9cbe46043c26 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:28.137428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee159f-278c-43e2-b7cb-75a7a8f103ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:28.138038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee159f-278c-43e2-b7cb-75a7a8f103ab service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:28.138323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee159f-278c-43e2-b7cb-75a7a8f103ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:28.138600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee159f-278c-43e2-b7cb-75a7a8f103ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:28.146016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee159f-278c-43e2-b7cb-75a7a8f103ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:28.147472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee159f-278c-43e2-b7cb-75a7a8f103ab service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:28.156254 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1179/2334] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:29:28.239654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd942910-fefc-47ff-9a18-c0ede3fc0ef7 None None] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:28.239941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd942910-fefc-47ff-9a18-c0ede3fc0ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:28.240218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd942910-fefc-47ff-9a18-c0ede3fc0ef7 None None] PATH_INFO: `/` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:28.241616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1156/2335] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 29 06:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 06:29:28.247174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:28.247413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:28.247696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:28.267787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] MFA Rules not processed for user `bbe64416672b408f8cdf93220cc31ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:28.269022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:28.279037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1180/2336] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 29 06:29:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:29:28.289239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd942910-fefc-47ff-9a18-c0ede3fc0ef7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:28.300697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd942910-fefc-47ff-9a18-c0ede3fc0ef7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:28.304266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3592ee7-9506-41f3-b968-432c0e933309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:28.304915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592ee7-9506-41f3-b968-432c0e933309 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:28.305194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592ee7-9506-41f3-b968-432c0e933309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:28.305465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592ee7-9506-41f3-b968-432c0e933309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:28.315522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3592ee7-9506-41f3-b968-432c0e933309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:28.316992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3592ee7-9506-41f3-b968-432c0e933309 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:28.328166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1157/2337] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:29:32.961714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:32.963137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:33.036317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd48c-180c-4759-84d8-9805e5e75f1f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:33.041295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:33.042593 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:33.042876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:33.043153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:33.044964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:33.058327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:33.059899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:33.069891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1181/2338] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:29:38.323639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=55 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:38.797381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff034-ed57-4e93-b342-be4b546470b5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:38.797671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff034-ed57-4e93-b342-be4b546470b5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:38.797949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff034-ed57-4e93-b342-be4b546470b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:38.817349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-80eff034-ed57-4e93-b342-be4b546470b5 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:38.818565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-80eff034-ed57-4e93-b342-be4b546470b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:38.884684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1158/2339] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 06:29:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:38.905481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:38.907034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:38.976161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484cfc-c1bc-498d-9bbd-7dc9eeb4d042 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:38.981369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436411c-6f5e-4dea-9f96-2c416c679d73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:38.981991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436411c-6f5e-4dea-9f96-2c416c679d73 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:38.982278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436411c-6f5e-4dea-9f96-2c416c679d73 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:38.982561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436411c-6f5e-4dea-9f96-2c416c679d73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:38.990872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436411c-6f5e-4dea-9f96-2c416c679d73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:38.992462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436411c-6f5e-4dea-9f96-2c416c679d73 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:39.001265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1182/2340] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:29:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:29:40.846918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eff034-ed57-4e93-b342-be4b546470b5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:40.955901 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eff034-ed57-4e93-b342-be4b546470b5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:40.961630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:40.962184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:40.962567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:40.962946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] PATH_INFO: `/v3/users/f1bf0e9a2117423aa3372e025a131e9c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:40.964955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1bf0e9a2117423aa3372e025a131e9c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:40.967570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:41.061219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1bf0e9a2117423aa3372e025a131e9c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.062120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1bf0e9a2117423aa3372e025a131e9c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.081580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1bf0e9a2117423aa3372e025a131e9c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.100795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1bf0e9a2117423aa3372e025a131e9c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.101924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1bf0e9a2117423aa3372e025a131e9c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.137285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1159/2341] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:29:40 2022] DELETE /identity/v3/users/f1bf0e9a2117423aa3372e025a131e9c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:41.141490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:41.142149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:41.142149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:41.192794 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:29:41.194042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:41.253654 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:41.267644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1183/2342] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:29:41.278814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:41.280392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:41.312124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:41.344321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d58ff4-6465-4912-aaf3-eb097528cd8c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:41.347972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:41.348629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:41.348918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:41.349199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:41.358636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:41.360061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:41.370135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1160/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:29:41.690220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:29:41.694088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:29:41.750581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc41c5d-6752-4c0c-9eeb-a2807cf83cd5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:29:41.758359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:29:41.759468 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:29:41.759842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:29:41.763857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] PATH_INFO: `/v3/projects/b6d476aafa4a4ed7964c35cef261021f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:29:41.763857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d476aafa4a4ed7964c35cef261021f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:29:41.765564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:29:41.840344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d476aafa4a4ed7964c35cef261021f', 'request_id': 'req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.848897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d476aafa4a4ed7964c35cef261021f', 'request_id': 'req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.856559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d476aafa4a4ed7964c35cef261021f', 'request_id': 'req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.873275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] The token cache is being invalidate because project b6d476aafa4a4ed7964c35cef261021f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:29:41.873575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.874273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-736a5117-bb5c-413b-ad37-2a20d9f7a6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:29:41.900606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1184/2344] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:29:41 2022] DELETE /identity/v3/projects/b6d476aafa4a4ed7964c35cef261021f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:29:51.365221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=15 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:29:51.865760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:30:02.297827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:02.299547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:02.322818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b89eb6-9e08-4bd3-81c0-601014e19118 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:02.409818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:02.410883 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:02.411312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:02.411598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] PATH_INFO: `/v3/users/32968493c6764a09b1f68cd6f4c3acfc` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:02.414888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32968493c6764a09b1f68cd6f4c3acfc)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:02.418813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:02.497438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32968493c6764a09b1f68cd6f4c3acfc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:02.498453 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32968493c6764a09b1f68cd6f4c3acfc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:02.521552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32968493c6764a09b1f68cd6f4c3acfc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:02.549294 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32968493c6764a09b1f68cd6f4c3acfc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:02.550435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32968493c6764a09b1f68cd6f4c3acfc'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:02.579921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1161/2345] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:30:02 2022] DELETE /identity/v3/users/32968493c6764a09b1f68cd6f4c3acfc => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:30:02.585342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:02.585986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:02.586079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:02.613076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:30:02.614357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:02.712258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1185/2346] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:30:02.724380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:02.725984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:02.802975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f112904-a2ac-475e-a046-b6f1c0110524 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:02.807570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac82652-ed37-43d0-971f-b99bc280903c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:02.808206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac82652-ed37-43d0-971f-b99bc280903c service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:02.808492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac82652-ed37-43d0-971f-b99bc280903c service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:02.808804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac82652-ed37-43d0-971f-b99bc280903c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:02.818170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac82652-ed37-43d0-971f-b99bc280903c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:02.819596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac82652-ed37-43d0-971f-b99bc280903c service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:02.830496 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1162/2347] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:30:03.098237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.099699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:03.112342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626327af-b6da-4ddb-892b-a5f835612bec None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.116931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.117568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.117851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.118123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] PATH_INFO: `/v3/projects/6e49e72350164b29b0c82de9fd0c714f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.119302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e49e72350164b29b0c82de9fd0c714f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.121259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.172676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e49e72350164b29b0c82de9fd0c714f', 'request_id': 'req-909678b1-8bea-42b8-a1c6-f33bc796932c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:03.180156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e49e72350164b29b0c82de9fd0c714f', 'request_id': 'req-909678b1-8bea-42b8-a1c6-f33bc796932c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:03.187299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e49e72350164b29b0c82de9fd0c714f', 'request_id': 'req-909678b1-8bea-42b8-a1c6-f33bc796932c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:03.198711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] The token cache is being invalidate because project 6e49e72350164b29b0c82de9fd0c714f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:30:03.199004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:03.199655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-909678b1-8bea-42b8-a1c6-f33bc796932c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:03.201219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1186/2348] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:30:03 2022] DELETE /identity/v3/projects/6e49e72350164b29b0c82de9fd0c714f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:30:03.206027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3c78-8c65-4799-9203-59a9aa3fffcb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.206326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3c78-8c65-4799-9203-59a9aa3fffcb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.206612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3c78-8c65-4799-9203-59a9aa3fffcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.227034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-011d3c78-8c65-4799-9203-59a9aa3fffcb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:30:03.228840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-011d3c78-8c65-4799-9203-59a9aa3fffcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:03.248433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:30:03.314917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1163/2349] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:30:03.383701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac3cfe-1c00-4eb3-9716-d3e4d172e1be None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.383992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac3cfe-1c00-4eb3-9716-d3e4d172e1be None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.384857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac3cfe-1c00-4eb3-9716-d3e4d172e1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.406188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-58ac3cfe-1c00-4eb3-9716-d3e4d172e1be None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:30:03.408499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-58ac3cfe-1c00-4eb3-9716-d3e4d172e1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:03.417717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1187/2350] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:30:03.421753 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3c78-8c65-4799-9203-59a9aa3fffcb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.432209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3c78-8c65-4799-9203-59a9aa3fffcb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.436737 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.437352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.437628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.437904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.438595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:30:03.438904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.440502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.453617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1164/2351] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:30:03.458321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac3cfe-1c00-4eb3-9716-d3e4d172e1be None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.468682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac3cfe-1c00-4eb3-9716-d3e4d172e1be None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.475756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.476389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.476717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.477000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.477682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.479875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.512629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1188/2352] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:30:03 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:30:03.516772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.543275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a3322-6b80-4724-a466-e31ffe7c1ddf admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.604923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.605936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.606367 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.606876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.607563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.609656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.635835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1165/2353] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:30:03 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:30:03.639941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.651535 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492936d-3ed7-4722-b663-0c1edfcd03fb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.657525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.658376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.658723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.659027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.659857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.661638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.677970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1189/2354] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:30:03.682820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.698540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da751060-358a-4d06-940a-28458f02d012 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.708755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075b3f9-dc4d-4a55-b662-a914e2bb3d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.709624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075b3f9-dc4d-4a55-b662-a914e2bb3d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.709916 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075b3f9-dc4d-4a55-b662-a914e2bb3d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.710199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075b3f9-dc4d-4a55-b662-a914e2bb3d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.710982 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075b3f9-dc4d-4a55-b662-a914e2bb3d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.713310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075b3f9-dc4d-4a55-b662-a914e2bb3d84 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.724923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1166/2355] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:30:03.730830 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.741703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb01650-300e-4173-930c-bc8428eecddf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.747180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.747860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.748147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.748428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] PATH_INFO: `/v3/projects/e9828753529b449d8e5feb9ae83afbd8/users/ecb3503efd854b7cbf92996afa1c3924/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.764238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9828753529b449d8e5feb9ae83afbd8, user_id=ecb3503efd854b7cbf92996afa1c3924, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.766931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.780764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1190/2356] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:30:03 2022] PUT /identity/v3/projects/e9828753529b449d8e5feb9ae83afbd8/users/ecb3503efd854b7cbf92996afa1c3924/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:30:03.791146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c4499-c252-4990-af98-315a18699ea9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.791422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c4499-c252-4990-af98-315a18699ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.791706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c4499-c252-4990-af98-315a18699ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.819966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-034c4499-c252-4990-af98-315a18699ea9 None None] MFA Rules not processed for user `ecb3503efd854b7cbf92996afa1c3924`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:30:03.821302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-034c4499-c252-4990-af98-315a18699ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:03.890943 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1167/2357] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:30:03.902525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:03.904306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:03.926548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:30:03.964655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3cff0-2aab-4931-baf3-4c4e6c4ad5fd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:03.968166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8129187-3ba4-45df-8bde-9e7ce2426492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:03.968816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129187-3ba4-45df-8bde-9e7ce2426492 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:03.969100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129187-3ba4-45df-8bde-9e7ce2426492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:03.969378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129187-3ba4-45df-8bde-9e7ce2426492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:03.977613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8129187-3ba4-45df-8bde-9e7ce2426492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:03.979866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8129187-3ba4-45df-8bde-9e7ce2426492 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:03.988889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1191/2358] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:30:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:30:06.807465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:06.808009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:06.808103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:06.827114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] MFA Rules not processed for user `5e05c9bdfc114478b5d80f077b970d17`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:30:06.828315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:06.841219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:30:06.889360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1168/2359] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:30:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:30:11.771118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad7a6-84d5-47a4-bde8-557a4bbad4a3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:11.772042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad7a6-84d5-47a4-bde8-557a4bbad4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:11.772476 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad7a6-84d5-47a4-bde8-557a4bbad4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:11.808009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b55ad7a6-84d5-47a4-bde8-557a4bbad4a3 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:30:11.809286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b55ad7a6-84d5-47a4-bde8-557a4bbad4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:11.945140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1192/2360] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 29 06:30:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 06:30:11.954609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:11.956095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:12.109959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee3c09-41fd-4029-b824-ae932b88aa83 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:12.121568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9181946-0ffe-441e-8f9e-2dfd0e81c019 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:12.122223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9181946-0ffe-441e-8f9e-2dfd0e81c019 service swift] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:12.122512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9181946-0ffe-441e-8f9e-2dfd0e81c019 service swift] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:12.122793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9181946-0ffe-441e-8f9e-2dfd0e81c019 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:12.135217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9181946-0ffe-441e-8f9e-2dfd0e81c019 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:12.136894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9181946-0ffe-441e-8f9e-2dfd0e81c019 service swift] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:12.146308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1169/2361] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 06:30:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 06:30:17.026264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ad7a6-84d5-47a4-bde8-557a4bbad4a3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:17.122496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ad7a6-84d5-47a4-bde8-557a4bbad4a3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:17.129986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:17.131059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:17.131812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:17.132261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] PATH_INFO: `/v3/users/ecb3503efd854b7cbf92996afa1c3924` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:17.134698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb3503efd854b7cbf92996afa1c3924)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:17.137849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:17.213548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb3503efd854b7cbf92996afa1c3924'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.234425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb3503efd854b7cbf92996afa1c3924'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.247276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:30:17.250052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb3503efd854b7cbf92996afa1c3924'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.261237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb3503efd854b7cbf92996afa1c3924'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.262105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb3503efd854b7cbf92996afa1c3924'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.268897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1193/2362] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:30:17 2022] DELETE /identity/v3/users/ecb3503efd854b7cbf92996afa1c3924 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:30:17.273578 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3f444-9cd4-470f-808e-2b36013efd15 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:17.274057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3f444-9cd4-470f-808e-2b36013efd15 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:17.274906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3f444-9cd4-470f-808e-2b36013efd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:17.398194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:17.400295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:17.426075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-4bf3f444-9cd4-470f-808e-2b36013efd15 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:30:17.427307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4bf3f444-9cd4-470f-808e-2b36013efd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:17.460448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a77a4b-3fc0-4421-bdf1-03d17e798b22 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:17.466489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:17.467418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:17.467774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:17.468124 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:17.470083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:17.499238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1170/2363] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:30:17.506043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3f444-9cd4-470f-808e-2b36013efd15 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:17.519947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3f444-9cd4-470f-808e-2b36013efd15 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:17.524984 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:17.525783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:17.526132 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:17.526492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:17.540967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:17.543399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:17.560720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1171/2364] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:30:17.578647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:17.581055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:17.596096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1194/2365] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:30:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:30:17.885181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:30:17.886887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:30:17.902195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b612-2381-4f18-bc4c-8fe3dd08432e service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:30:17.907286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:30:17.907609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:30:17.907891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:30:17.908172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] PATH_INFO: `/v3/projects/e9828753529b449d8e5feb9ae83afbd8` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:30:17.910072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9828753529b449d8e5feb9ae83afbd8)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:30:17.912173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:30:17.953605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:30:17.967408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9828753529b449d8e5feb9ae83afbd8', 'request_id': 'req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.973999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9828753529b449d8e5feb9ae83afbd8', 'request_id': 'req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.981918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9828753529b449d8e5feb9ae83afbd8', 'request_id': 'req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.992776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] The token cache is being invalidate because project e9828753529b449d8e5feb9ae83afbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:30:17.993650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.994352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-9d516c36-90b7-42a4-8c54-1daf0a3dbd1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:30:17.996122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1172/2366] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:30:17 2022] DELETE /identity/v3/projects/e9828753529b449d8e5feb9ae83afbd8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:30:27.589199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=2,keystone:SELECT=53,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:30:27.989835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:31:27.231270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:31:27.245325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70bbad-9e51-4d00-8233-0c94c9e2ac77 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:31:27.310219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:31:27.310490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:27.310871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:27.311241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:27.313195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:27.402562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:31:27.405817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:31:27.430065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1195/2367] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:31:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:31:28.126545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:28.127085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:28.127085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:28.145742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] MFA Rules not processed for user `e1b9767f26254a6cba945f4cae562b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:31:28.146933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:28.200711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1173/2368] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:31:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:31:28.595859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:31:28.597395 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:28.608869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f6e8f-96b7-4efd-a693-584e6483ca81 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:31:28.612574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6a464-0963-4952-a482-a64af19bf33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:31:28.613185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6a464-0963-4952-a482-a64af19bf33b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:28.613461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6a464-0963-4952-a482-a64af19bf33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:28.613735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6a464-0963-4952-a482-a64af19bf33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:28.615438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-82f6a464-0963-4952-a482-a64af19bf33b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:28.630891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6a464-0963-4952-a482-a64af19bf33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:31:28.632333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6a464-0963-4952-a482-a64af19bf33b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:31:28.646246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1196/2369] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:31:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:31:35.763642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:31:35.765168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:35.783271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc909a3-2b8a-450a-a4ac-8bb15f7609dc None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:31:35.789202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:31:35.789838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:35.790205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:35.790488 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:35.793047 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:35.861419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:31:35.862828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:31:35.876715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1174/2370] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:31:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:31:38.639444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=60 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:31:39.563291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:39.563584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:39.563872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:39.583454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] MFA Rules not processed for user `b34e3e2e2e6049428295588959ca595c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:31:39.584696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:39.592068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1197/2371] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:31:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:31:40.355556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:31:40.366747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fea76c-8121-4793-af42-766d661f0a9f service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:31:40.373013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:31:40.373635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:40.373915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:40.374191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:40.375980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:40.387701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:31:40.389475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:31:40.399568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1175/2372] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:31:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:31:44.753533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:31:44.755031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:44.767229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e455cf-ec71-4362-8d61-9b85f457f1ad None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:31:44.770938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:31:44.771551 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:44.771827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:44.772095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:44.774039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:44.784675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:31:44.786100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:31:44.794704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1198/2373] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:31:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:31:49.887705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:31:49.889104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:49.903818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a062329-8d46-4e02-b9a2-df13dbb61fea service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:31:49.909535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:31:49.910174 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:31:49.910458 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:31:49.910733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:31:49.912492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:31:49.971104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:31:49.972748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:31:49.981991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1176/2374] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:31:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:31:54.790517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=9 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:31:59.977572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=87 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:32:35.731953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:32:35.748079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde35e4-6ec5-43c9-add8-2a0ad6716ec6 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:32:35.753648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:32:35.754269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:32:35.754555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:32:35.754835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:32:35.756846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:32:35.770438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:32:35.772069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:32:35.783243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1199/2375] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:32:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:32:45.778701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:33:24.282090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:33:24.285320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:33:24.342544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b92b40-9333-470d-ba87-7891f73e2b15 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:33:24.346136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:33:24.346758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:33:24.347035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:33:24.347306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:33:24.349037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:33:24.411734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:33:24.413187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:33:24.422330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1177/2376] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:33:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:33:34.418584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=48 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:34:13.732255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:34:13.743748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d7906-c962-4a5b-b9d2-d1e71f2a7cdc service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:34:13.750459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:34:13.751079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:34:13.751394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:34:13.751674 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:34:13.753474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:34:13.812497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:34:13.814121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:34:13.823532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1200/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:34:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:34:23.818910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=28 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:34:36.833669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:34:36.835034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:34:36.984298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9efb-4b71-4fd7-a4de-68a6e0a7e31b service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:34:36.989446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:34:36.990056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:34:36.990332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:34:36.990603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:34:36.992323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:34:37.004206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:34:37.005812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:34:37.015322 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1178/2378] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:34:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:34:47.011212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=63 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:36:28.759910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:36:28.789076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38183ca-9d17-44bd-abc7-b9eed088cdcf service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:36:28.793543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:36:28.794161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:36:28.794448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:36:28.794727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:36:28.811523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:36:28.812994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:36:28.834154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1201/2379] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:36:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:36:30.006655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:36:30.016514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934a07c-9053-40ba-bd45-7918cb4d8e3d service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:36:30.020060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:36:30.020701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:36:30.020983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:36:30.021253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:36:30.031544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:36:30.032983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:36:30.043775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1179/2380] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:36:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:36:36.871204 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:36:36.882422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8709b-06ac-40be-ab81-d0b26acaf822 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:36:36.888119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:36:36.888785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:36:36.889070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:36:36.889345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:36:36.897185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:36:36.898750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:36:36.925716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1202/2381] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:36:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:36:40.038924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:36:46.923128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=35 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:36:49.146556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:36:49.158191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3a307-2c2e-4d92-a4b4-68cace72fd20 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:36:49.163630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:36:49.164227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:36:49.164506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:36:49.164807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:36:49.173183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:36:49.174729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:36:49.201795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1180/2382] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:36:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:36:59.198986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=13 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:37:24.625270 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:37:24.645152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e891a0f-0473-42ff-bd86-ff0fd9de1f7a service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:37:24.648744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:37:24.649970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:37:24.650255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:37:24.650529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:37:24.659183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:37:24.660652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:37:24.687514 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1203/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:37:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:37:34.684928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=17 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:37:39.561022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:37:39.570198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac4c8-6949-4a0d-9f9b-a09a71730005 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:37:39.573879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:37:39.574474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:37:39.574749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:37:39.575017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:37:39.582650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:37:39.584055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:37:39.592555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1181/2384] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:37:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:37:49.589117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:38:20.351731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:38:20.365008 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08798be3-553a-4d48-b231-49187cb4167a service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:38:20.371508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:38:20.372134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:38:20.372443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:38:20.372756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:38:20.374584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:38:20.399973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:38:20.401606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:38:20.412302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1204/2385] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:38:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:38:24.693007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:38:24.705910 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8db933-c4b0-426a-8672-ce46f565ba46 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:38:24.712353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:38:24.712998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:38:24.713280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:38:24.713552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:38:24.721356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:38:24.722954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:38:24.751395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1182/2386] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:38:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:38:30.407754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=11 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:38:34.748574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=13 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:39:24.860921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:39:24.875308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda47c2-c944-4d73-b3ed-d6c55eac625e service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:39:24.893909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:39:24.893909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:39:24.893909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:39:24.893909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:39:24.902148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:39:24.904378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:39:24.948371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1205/2387] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:39:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:39:34.945768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=13 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:39:43.418341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:39:43.430188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ccf86-9b62-4a62-8fd0-401398289e96 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:39:43.435176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:39:43.435814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:39:43.436094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:39:43.436369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:39:43.444736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:39:43.446270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:39:43.455884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1183/2388] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:39:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:39:53.451894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:41:28.655510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:41:28.709844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671017e-f5b5-4169-9394-6b76e90f33f4 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:41:28.713839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:41:28.714457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:41:28.714744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:41:28.715026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:41:28.716953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:41:28.787865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:41:28.789324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:41:28.800677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1206/2389] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:41:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:41:29.977951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:41:29.979522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:41:30.005728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4464213-c989-47ce-93c8-9a9527adb579 service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:41:30.010112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:41:30.010726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:41:30.011010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:41:30.011287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:41:30.013221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:41:30.026538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:41:30.027944 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:41:30.038755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1184/2390] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:41:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:41:38.795917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=54 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:41:40.034009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=11 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:42:32.091062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:42:32.092597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:42:32.118733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708c819-ad7b-442b-b2ee-3b49b6680669 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:42:32.122653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:42:32.123279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:42:32.123568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:42:32.123847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:42:32.125795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:42:32.200292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:42:32.201781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:42:32.211387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1207/2391] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:42:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:42:32.256026 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:42:32.265611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5a324-c4e4-4e78-aefe-d3da1e0a6d0e service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:42:32.269157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:42:32.269758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:42:32.270039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:42:32.270573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:42:32.272504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:42:32.362368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:42:32.363818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:42:32.374274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1185/2392] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:42:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:42:32.428736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:42:32.438636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920063a-75fc-40be-9cdd-b14c30554763 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:42:32.442510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:42:32.443189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:42:32.443557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:42:32.443901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:42:32.445897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:42:32.519555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:42:32.521154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:42:32.532376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1208/2393] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:42:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:42:42.369534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=33 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:42:42.527285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=71 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:43:25.666664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:43:25.668494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:43:25.725786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534244-9acf-4332-b505-952bdd26e79a service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:43:25.730191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:43:25.730832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:43:25.731116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:43:25.731392 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:43:25.733345 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:43:25.809618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:43:25.811140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:43:25.820700 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1186/2394] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:43:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:43:27.145963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:43:27.165094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:43:27.184604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685267c-6597-4c5c-813d-47d88694c5ba service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:43:27.200356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:43:27.201487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:43:27.202017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:43:27.202528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:43:27.213347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:43:27.215268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:43:27.237056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1209/2395] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:43:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:43:35.817010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=53 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:43:37.221899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:43:47.127767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:43:47.129441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:43:47.144362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c6e06-ed70-4b05-994b-5225f76d5766 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:43:47.148373 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:43:47.149082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:43:47.149365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:43:47.149641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:43:47.151718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:43:47.161924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:43:47.163355 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:43:47.176213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1187/2396] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:43:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:43:57.168389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:44:20.717559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:44:20.728396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2362675-b060-4ddd-87fc-a58fa41283cf service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:44:20.732686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:44:20.733886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:44:20.734173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:44:20.734449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:44:20.736231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:20.820003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:44:20.821924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:44:20.841447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1210/2397] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:44:24.010429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:44:24.012041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:24.028872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eed3d4-608a-45e3-a2ff-d51b29f5aad4 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:44:24.034266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:44:24.035283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:44:24.035566 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:44:24.035839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:44:24.037759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:24.053062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:44:24.055479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:44:24.074264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1188/2398] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:44:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:44:27.081611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:44:27.083158 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:27.098433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74055-e445-4cb3-b654-5c0ec79b7e65 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:44:27.103408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:44:27.104082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:44:27.104353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:44:27.104675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] PATH_INFO: `/v3/users/1d436bd35f48454da2dc2e607247fe02` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:44:27.120782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d436bd35f48454da2dc2e607247fe02)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:44:27.122733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:44:27.172480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d436bd35f48454da2dc2e607247fe02'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.180312 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d436bd35f48454da2dc2e607247fe02'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.196711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d436bd35f48454da2dc2e607247fe02'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.205106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d436bd35f48454da2dc2e607247fe02'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.205750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d436bd35f48454da2dc2e607247fe02'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.212820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1211/2399] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:44:27 2022] DELETE /identity/v3/users/1d436bd35f48454da2dc2e607247fe02 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:44:27.380500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:44:27.391928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c18bb-9ac3-425e-991d-f0b7bae9a15d service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:44:27.455017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:44:27.455773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:44:27.456125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:44:27.456470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] PATH_INFO: `/v3/projects/6b5916906d6048bb96c13c457bf367e3` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:44:27.457733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5916906d6048bb96c13c457bf367e3)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:44:27.459692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:44:27.506885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5916906d6048bb96c13c457bf367e3', 'request_id': 'req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.513948 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5916906d6048bb96c13c457bf367e3', 'request_id': 'req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.527288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5916906d6048bb96c13c457bf367e3', 'request_id': 'req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.537570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] The token cache is being invalidate because project 6b5916906d6048bb96c13c457bf367e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:44:27.537853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.538464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.540525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1189/2400] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:44:27 2022] DELETE /identity/v3/projects/6b5916906d6048bb96c13c457bf367e3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:44:27.544373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:44:27.545839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:27.559506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300ec54-9294-4eef-b549-27a4b620a230 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:44:27.583368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:44:27.618399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:44:27.618974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:44:27.619252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:44:27.619525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] PATH_INFO: `/v3/users/648d19509ebe4616a24bd832fb762872` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:44:27.632755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648d19509ebe4616a24bd832fb762872)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:44:27.637435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:44:27.683629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648d19509ebe4616a24bd832fb762872'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.690481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648d19509ebe4616a24bd832fb762872'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.702538 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648d19509ebe4616a24bd832fb762872'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.709071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648d19509ebe4616a24bd832fb762872'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.709788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648d19509ebe4616a24bd832fb762872'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:27.716666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1212/2401] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:44:27 2022] DELETE /identity/v3/users/648d19509ebe4616a24bd832fb762872 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:44:27.898033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:44:27.899869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:27.917540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee7f4e-c8c6-4e2a-a42f-f6c36fe93cb2 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:44:27.980431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:44:27.981255 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:44:27.981569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:44:27.981845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] PATH_INFO: `/v3/projects/2f8be70807e2436e840ac66dca5a11d0` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:44:27.992515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8be70807e2436e840ac66dca5a11d0)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:44:27.994721 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:44:28.004953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:44:28.042184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8be70807e2436e840ac66dca5a11d0', 'request_id': 'req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:28.048491 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8be70807e2436e840ac66dca5a11d0', 'request_id': 'req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:28.056181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8be70807e2436e840ac66dca5a11d0', 'request_id': 'req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:28.066471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] The token cache is being invalidate because project 2f8be70807e2436e840ac66dca5a11d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:44:28.066743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:28.067364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:44:28.068882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1190/2402] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:44:27 2022] DELETE /identity/v3/projects/2f8be70807e2436e840ac66dca5a11d0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:44:37.712935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=46,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:44:38.063600 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:44:44.824905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:44:44.826302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:44.953442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e485129-5aad-46d8-b82a-db206c420836 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:44:45.011925 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:44:45.012470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:44:45.012790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:44:45.013086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:44:45.014790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:44:45.025784 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:44:45.027343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:44:45.036974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1213/2403] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:44:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:44:55.032955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=84 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:45:26.015704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:45:26.017214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:45:26.077248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5de56f-ea5b-410c-9b2c-bf7b129e9e01 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:45:26.081067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:45:26.081692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:45:26.081969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:45:26.082244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:45:26.083991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] Could not find project: 6b5916906d6048bb96c13c457bf367e3.: keystone.exception.ProjectNotFound: Could not find project: 6b5916906d6048bb96c13c457bf367e3. Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 29 06:45:26.097161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:45:26.098310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 06:45:26.099427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application with Lock( Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._enter() Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.creator() Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application created_value = creator( Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 06:45:26.101362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 06:45:26.102484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6b5916906d6048bb96c13c457bf367e3. Jul 29 06:45:26.102484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: ERROR keystone.server.flask.application Jul 29 06:45:26.102484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1191/2404] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:45:26 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 83 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 29 06:45:36.088110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=24 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:46:28.288239 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:46:28.289664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:46:28.345738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d42bd-b2ed-4efa-92aa-fb37fcb46b5a service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:46:28.349391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:46:28.349998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:46:28.350286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:46:28.350566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:46:28.352327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:46:28.427072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:46:28.428512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:46:28.441700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1214/2405] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:46:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:46:29.605798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:46:29.607214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:46:29.639300 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaeaa-516c-4688-a65f-8774355f92d6 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:46:29.642922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:46:29.643528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:46:29.643814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:46:29.644091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:46:29.645827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:46:29.665183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:46:29.666777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:46:29.687385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1192/2406] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:46:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:46:38.436509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=54 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:46:39.684852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=22 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:47:32.773300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:47:32.774720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:47:32.804178 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:47:32.805639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:47:32.815122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdc3b8-a68b-4718-b089-1b4c60514213 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:47:32.819494 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c3a2c-a6b8-4d8d-a3c9-854de0d7f287 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:47:32.821343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:47:32.821953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:47:32.822236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:47:32.822510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:47:32.824881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:47:32.825800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:47:32.826409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:47:32.826690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:47:32.826964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:47:32.828686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:47:32.897505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:47:32.899139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:47:32.907009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:47:32.908627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:47:32.918412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1215/2407] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:47:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:47:32.928755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1193/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:47:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:47:42.915636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=48 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:47:42.925507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=36 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:48:26.039300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:48:26.051145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7db8-d48b-47a6-96fc-086f03581fe1 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:48:26.056760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:48:26.058128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:48:26.058421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:48:26.058704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:48:26.060467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:48:26.129473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:48:26.131074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:48:26.149795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1216/2409] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:48:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:48:33.233025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:48:33.234552 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:48:33.256726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00175e4-4eef-45d0-a7d1-0072d4f3eb31 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:48:33.260974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f2862-5a1a-46a7-837b-a73465a7f942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:48:33.261582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2862-5a1a-46a7-837b-a73465a7f942 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:48:33.261862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2862-5a1a-46a7-837b-a73465a7f942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:48:33.262226 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2862-5a1a-46a7-837b-a73465a7f942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:48:33.263968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-356f2862-5a1a-46a7-837b-a73465a7f942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:48:33.334720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f2862-5a1a-46a7-837b-a73465a7f942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:48:33.336178 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f2862-5a1a-46a7-837b-a73465a7f942 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:48:33.355106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1194/2410] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:48:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:48:36.146948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=36 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:48:43.352230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=40 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:48:53.111243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:48:53.121378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbc81f-e7ce-486b-893b-4d68ca1295b8 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:48:53.125001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:48:53.125618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:48:53.125902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:48:53.126180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:48:53.127999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:48:53.139138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:48:53.140601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:48:53.149028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1217/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:48:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:49:03.145308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:49:28.077342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:49:28.077914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:49:28.077914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:49:28.105342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] MFA Rules not processed for user `217b321936224e2ba928b983c4e40017`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:49:28.106530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:49:28.113261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1195/2412] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:49:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:49:28.417701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:49:28.426367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8191-d5e3-4f31-90c2-15948ce16dd9 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:49:28.430018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:49:28.431272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:49:28.431557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:49:28.431866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:49:28.433767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:49:28.443845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:49:28.445298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:49:28.453394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1218/2413] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:49:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:49:28.779102 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:49:28.780516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:49:28.795291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f3f24-71e5-49d7-b03a-a8cc301792fe None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:49:28.800219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:49:28.800865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:49:28.801146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:49:28.801421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:49:28.803143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:49:28.816286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:49:28.817306 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:49:28.832702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1196/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:49:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:49:31.250569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:49:31.269322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269edf5-a6c8-4c82-825c-4d463b2523a5 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:49:31.276829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:49:31.277813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:49:31.278100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:49:31.278376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:49:31.280283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:49:31.310412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:49:31.312051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:49:31.324377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1219/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:49:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:49:35.115071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:49:35.130289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dc4fd-19f0-43fe-af6e-f555a74d164b service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:49:35.136907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:49:35.137529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:49:35.137809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:49:35.138080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:49:35.139824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:49:35.153876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:49:35.155464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:49:35.168633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1197/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:49:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:49:41.318914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=17 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:49:45.163833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=18 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:49:51.828245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:49:51.840668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6c60-db8b-4faa-bb4a-ddcf539cd183 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:49:51.845832 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:49:51.846454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:49:51.846740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:49:51.847017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:49:51.856590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:49:51.858243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:49:51.869530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1220/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:49:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:50:01.864218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:51:29.323689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:51:29.332873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4c27-ac23-4ab9-b292-b63d5fc94cfd service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:51:29.336678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:51:29.337277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:51:29.337560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:51:29.337832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:51:29.348055 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:51:29.349485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:51:29.360885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1198/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:51:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:51:30.535016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:51:30.544663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4219-9a5d-4b56-8f77-581c5ac70a8a service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:51:30.548178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:51:30.548824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:51:30.549106 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:51:30.549384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:51:30.560013 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:51:30.561489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:51:30.572829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1221/2419] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:51:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:51:39.355999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:51:40.567504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:52:33.770459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:52:33.779323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13183d-ab1a-4683-845d-8e5951b9a2f9 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:52:33.782872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:52:33.783470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:52:33.783748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:52:33.784018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:52:33.792387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:52:33.793816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:52:33.802498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1199/2420] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:52:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:52:33.881765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:52:33.892351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65263c-4cd7-4e05-9e55-995215ded182 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:52:33.895887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:52:33.896483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:52:33.896826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:52:33.897104 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:52:33.906020 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:52:33.907421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:52:33.916230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1222/2421] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:52:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:52:43.798452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:52:43.912180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:53:25.015278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:25.027553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a66e1-1db0-4d2d-a72c-c52834e951b1 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:25.033140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:25.033779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:25.034060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:25.034444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:25.042812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:25.044403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:25.053181 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1200/2422] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:53:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:53:27.218935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:27.228931 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274001a-ed2c-40f0-be79-dcead493fd45 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:27.232703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:27.233324 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:27.233611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:27.233899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:27.235638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:27.298088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:27.299517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:27.309776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1223/2423] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:53:30.300272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:30.314434 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6dab-98a9-4aa8-9285-ee4e5e57fba2 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:30.319542 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:30.320855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:30.321153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:30.321603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:30.323777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:30.336436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:30.338326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:30.351710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1201/2424] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:53:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:53:30.361851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:30.363429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:30.380038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ddf6-92ab-4675-a7f5-ff0f2734c23c service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:30.385581 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:30.386331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:30.386679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:30.387028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] PATH_INFO: `/v3/users/e1b9767f26254a6cba945f4cae562b70` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:30.402645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b9767f26254a6cba945f4cae562b70)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:30.404713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:30.454618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b9767f26254a6cba945f4cae562b70'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.461847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b9767f26254a6cba945f4cae562b70'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.475133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b9767f26254a6cba945f4cae562b70'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.483220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b9767f26254a6cba945f4cae562b70'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.483841 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b3aecbfe-ad5c-4933-9a04-cbdc316ea130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b9767f26254a6cba945f4cae562b70'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.489965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1224/2425] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 06:53:30 2022] DELETE /identity/v3/users/e1b9767f26254a6cba945f4cae562b70 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:53:30.661715 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:30.673089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df566b8e-3f3f-446b-8bde-42aa8837014d service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:30.733917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:30.734563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:30.734839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:30.735113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] PATH_INFO: `/v3/projects/a3b773b1bba540f79c21b042d5cf91de` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:30.736250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b773b1bba540f79c21b042d5cf91de)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:30.738177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:30.783424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b773b1bba540f79c21b042d5cf91de', 'request_id': 'req-8187baf4-df21-4f67-83d4-cf35979f8b9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.789930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b773b1bba540f79c21b042d5cf91de', 'request_id': 'req-8187baf4-df21-4f67-83d4-cf35979f8b9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.796983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b773b1bba540f79c21b042d5cf91de', 'request_id': 'req-8187baf4-df21-4f67-83d4-cf35979f8b9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.807828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] The token cache is being invalidate because project a3b773b1bba540f79c21b042d5cf91de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 06:53:30.808177 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.808832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8187baf4-df21-4f67-83d4-cf35979f8b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 06:53:30.810319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1202/2426] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 06:53:30 2022] DELETE /identity/v3/projects/a3b773b1bba540f79c21b042d5cf91de => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:53:30.816138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8aa2b-518d-458f-8043-5735b3947a46 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:30.816434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8aa2b-518d-458f-8043-5735b3947a46 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:30.816770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8aa2b-518d-458f-8043-5735b3947a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:30.873547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3fd8aa2b-518d-458f-8043-5735b3947a46 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:53:30.874639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3fd8aa2b-518d-458f-8043-5735b3947a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:30.893229 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:53:30.935100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1225/2427] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:53:30.943153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a999d8-7ffb-46f1-91bf-d23777445168 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:30.943445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a999d8-7ffb-46f1-91bf-d23777445168 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:30.943730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a999d8-7ffb-46f1-91bf-d23777445168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:30.963498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-86a999d8-7ffb-46f1-91bf-d23777445168 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:53:30.964744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-86a999d8-7ffb-46f1-91bf-d23777445168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:30.972574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1203/2428] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:53:30.977648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8aa2b-518d-458f-8043-5735b3947a46 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:30.989790 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8aa2b-518d-458f-8043-5735b3947a46 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:30.995695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:30.996302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:30.996615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:30.996895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:30.997583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 06:53:30.997892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:30.999486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:31.011443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1226/2429] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 06:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 06:53:31.015691 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a999d8-7ffb-46f1-91bf-d23777445168 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:31.025452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a999d8-7ffb-46f1-91bf-d23777445168 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:31.030028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:31.030632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:31.030915 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:31.031192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:31.031843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:31.033783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:31.071623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1204/2430] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 06:53:31 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:53:31.075739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:31.086711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2f55b-12ba-4aef-9666-03b406ee09b6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:31.143653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:31.144176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:31.144458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:31.144761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:31.145411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:31.147317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:31.168560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1227/2431] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 06:53:31 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 06:53:31.172632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:31.183497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357b084-af16-4f75-b4e3-d91e5cd6236b admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:31.188189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:31.188827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:31.189111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:31.189459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:31.190152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:31.191739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:31.202237 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1205/2432] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 06:53:31.206041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:31.217310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2582e2e-dc3d-4f3b-b450-8fcdc6df0f2e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:31.222006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e38713-e1ab-4a1b-9da5-3b88f4c5468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:31.222616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e38713-e1ab-4a1b-9da5-3b88f4c5468e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:31.222904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e38713-e1ab-4a1b-9da5-3b88f4c5468e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:31.223184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e38713-e1ab-4a1b-9da5-3b88f4c5468e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:31.223868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e38713-e1ab-4a1b-9da5-3b88f4c5468e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:31.225504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e38713-e1ab-4a1b-9da5-3b88f4c5468e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:31.236343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1228/2433] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 06:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 06:53:31.240464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:31.250437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b5d9-f637-4add-8ddb-b0cae5ab7f64 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:31.254925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:31.255580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:31.255865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:31.256185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] PATH_INFO: `/v3/projects/10b54d7ff9644981893f886e59f22021/users/12a751b41fd546ac9017c67d6de4ebdd/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:31.268499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:53:31.271722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b54d7ff9644981893f886e59f22021, user_id=12a751b41fd546ac9017c67d6de4ebdd, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:31.275205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:31.288516 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1206/2434] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 06:53:31 2022] PUT /identity/v3/projects/10b54d7ff9644981893f886e59f22021/users/12a751b41fd546ac9017c67d6de4ebdd/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 06:53:31.293570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4fbc8-b734-4a3c-a1bc-843156446779 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:31.293862 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4fbc8-b734-4a3c-a1bc-843156446779 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:31.294137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4fbc8-b734-4a3c-a1bc-843156446779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:31.314415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b9e4fbc8-b734-4a3c-a1bc-843156446779 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:53:31.315605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b9e4fbc8-b734-4a3c-a1bc-843156446779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:31.373580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1229/2435] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:53:31.383665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:31.385333 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:31.453256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1a411-69e3-45f0-9acc-9b1fa9a0d895 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:31.456790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:31.457398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:31.457680 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:31.457953 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:31.467077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:31.468475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:31.477903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1207/2436] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:53:35.629568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63956d-1edb-4822-a8d0-74cb88e12134 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:35.630055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63956d-1edb-4822-a8d0-74cb88e12134 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:35.630155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63956d-1edb-4822-a8d0-74cb88e12134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:35.657591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3d63956d-1edb-4822-a8d0-74cb88e12134 None None] MFA Rules not processed for user `12a751b41fd546ac9017c67d6de4ebdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:53:35.658759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d63956d-1edb-4822-a8d0-74cb88e12134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:35.678051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:53:35.733544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1230/2437] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:53:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:53:35.749191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:35.750625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:35.829444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08387-dc6e-43fa-8dca-03d1133e4414 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:35.840619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:35.841258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:35.841546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:35.841961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:35.851165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:35.852894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:35.862258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1208/2438] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:53:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:53:37.262264 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:37.262770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:37.262890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:37.281622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] MFA Rules not processed for user `12a751b41fd546ac9017c67d6de4ebdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:53:37.282827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:37.289836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1231/2439] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:53:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:53:37.603079 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:37.604527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:37.616777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e670b7f-efd5-46b7-bab6-56f15a5a9a2b service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:37.620469 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:37.621111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:37.621398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:37.621676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:37.623524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:37.633977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:37.636844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:37.644154 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1209/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:53:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:53:47.276330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=21 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:53:47.640881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:53:49.570026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:53:49.571499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:49.587988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f7e69-496b-4ed2-be7a-3a037b0a6288 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:53:49.594364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:53:49.595068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:53:49.595422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:53:49.595799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:53:49.597611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:53:49.610572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:53:49.612180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:53:49.624152 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1232/2441] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:53:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 06:53:59.619553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:54:00.153667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:54:00.162958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fa9be-360c-4219-a2fc-9600bec4bb70 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:54:00.167108 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:54:00.167730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:54:00.168015 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:54:00.168315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:54:00.170271 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:54:00.180253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:54:00.181694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:54:00.189541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1210/2442] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:54:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:54:10.186066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:54:57.819766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:54:57.821192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:54:57.948341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da9bd7-d20e-4dca-9d8a-e1204e995995 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:54:57.953387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfc2cf-d362-4af3-9344-fa5e5a70f252 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:54:57.954001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfc2cf-d362-4af3-9344-fa5e5a70f252 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:54:57.954282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfc2cf-d362-4af3-9344-fa5e5a70f252 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:54:57.954557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfc2cf-d362-4af3-9344-fa5e5a70f252 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:54:57.956279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ecdfc2cf-d362-4af3-9344-fa5e5a70f252 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:54:57.968268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfc2cf-d362-4af3-9344-fa5e5a70f252 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:54:57.969859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfc2cf-d362-4af3-9344-fa5e5a70f252 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:54:57.980405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1233/2443] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:54:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:55:07.975523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=58 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:55:21.584435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:55:21.600766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47cdc8-a9cd-4857-bddc-e6e5e82d9853 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:55:21.610241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:55:21.610867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:55:21.611151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:55:21.611426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:55:21.613305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:55:21.626617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:55:21.628210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:55:21.640083 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1211/2444] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:55:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:55:24.828406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c7cda-ce9a-4364-a9da-598e8e75dc13 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:55:24.828987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c7cda-ce9a-4364-a9da-598e8e75dc13 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:55:24.828987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c7cda-ce9a-4364-a9da-598e8e75dc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:55:24.848147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aa8c7cda-ce9a-4364-a9da-598e8e75dc13 None None] MFA Rules not processed for user `12a751b41fd546ac9017c67d6de4ebdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 06:55:24.849393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa8c7cda-ce9a-4364-a9da-598e8e75dc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:55:24.857174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1234/2445] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 06:55:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 06:55:25.454884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:55:25.466612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530b51-edd2-4fbb-8cc8-5000b9f57538 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:55:25.471534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:55:25.472246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:55:25.472525 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:55:25.472825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:55:25.474632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:55:25.486481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:55:25.488041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:55:25.498926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1212/2446] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:55:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:55:32.785429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c7cda-ce9a-4364-a9da-598e8e75dc13 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:55:32.799638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c7cda-ce9a-4364-a9da-598e8e75dc13 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:55:32.805185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:55:32.805808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:55:32.806121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:55:32.806970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:55:32.808787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:55:32.823048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:55:32.824682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:55:32.836067 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1235/2447] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:55:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:55:35.493929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=12 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:55:42.831288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=9 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:57:28.341323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:57:28.361292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94545ef1-5d9a-4f37-969e-10c2d0d9d81a service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:57:28.364892 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:57:28.365504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:57:28.365790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:57:28.366068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:57:28.367819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:57:28.381102 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:57:28.382517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:57:28.393544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1213/2448] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:57:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:57:29.580250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:57:29.592195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e33e5-3d49-4177-9c0a-c90b8c3ef3ee service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:57:29.602760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:57:29.604179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:57:29.604465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:57:29.604780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:57:29.608530 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:57:29.626656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:57:29.628116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:57:29.641176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1236/2449] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:57:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:57:38.388555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=10 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:57:39.635498 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:58:34.724221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:58:34.725636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:58:34.810629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f12fb8-7997-4562-9023-54cc7d451edd service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:58:34.814299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:58:34.815486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:58:34.815769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:58:34.816044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:58:34.817820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:58:34.900824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:58:34.902262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:58:34.921130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1214/2450] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:58:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:58:34.986091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:58:35.007314 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12607515-d533-401d-ba4a-a4649cc8adcf service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:58:35.011009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:58:35.011628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:58:35.011938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:58:35.012215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:58:35.013967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:58:35.098704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:58:35.100146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:58:35.118753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1237/2451] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:58:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:58:35.166353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:58:35.175336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8c762-06d0-440d-9eb2-eadfe8f0682b service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:58:35.178881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:58:35.179481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:58:35.179767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:58:35.180040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:58:35.181740 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 06:58:35.235412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:58:35.262757 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:58:35.264171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:58:35.282778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1215/2452] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:58:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:58:35.343949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:58:35.352596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e72576-d34d-48e4-a650-ff9d3fc0f9b5 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:58:35.356325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:58:35.356996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:58:35.357287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:58:35.357570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:58:35.367028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:58:35.368455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:58:35.378109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1238/2453] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:58:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:58:45.280302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=16 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:58:45.373333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=52 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 06:59:04.052595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 06:59:04.061877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f80c0-b274-48b5-87f7-9cb54b65534d service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 06:59:04.066347 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 06:59:04.066969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 06:59:04.067252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 06:59:04.067528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 06:59:04.074868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 06:59:04.076277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 06:59:04.084159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1216/2454] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 06:59:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 06:59:14.080854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=6 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:00:02.798567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:00:02.811060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef8758-1d0f-4936-a261-66ef5cd1f036 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:00:02.816142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:00:02.816822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:00:02.817110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:00:02.817389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:00:02.826697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:00:02.828475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:00:02.838903 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1239/2455] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:00:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:00:12.834391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:24.225733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:24.296233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845c2b3-6c2f-4461-9d1d-10e3bc77744f service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:24.307976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:24.308676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:24.309020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:24.309307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:24.319122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:24.320720 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:24.331265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1217/2456] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:25.384156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:25.395231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08daf9-b365-4e08-b2db-8f4507741654 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:25.399129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:25.400063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:25.400633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:25.401166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:25.403142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:25.462463 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:25.464277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:25.496567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1240/2457] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:26.201852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:26.203303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:26.218945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f987d5-232b-40bc-b4c1-d323856131fd service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:26.225164 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:26.225713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:26.225999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:26.226280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:26.227991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:26.239774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:26.241375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:26.252240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1218/2458] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:26.265554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:26.268279 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:26.287450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a425b5-555e-45a2-b26d-44a09ab686c0 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:26.292740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:26.293401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:26.293747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:26.294153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] PATH_INFO: `/v3/users/5e05c9bdfc114478b5d80f077b970d17` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:26.295192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e05c9bdfc114478b5d80f077b970d17)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:26.297164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:26.346321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e05c9bdfc114478b5d80f077b970d17'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.354750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e05c9bdfc114478b5d80f077b970d17'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.375028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e05c9bdfc114478b5d80f077b970d17'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.384193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e05c9bdfc114478b5d80f077b970d17'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.384986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0a9dfd97-8912-4cb5-963e-d0b6de28a9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e05c9bdfc114478b5d80f077b970d17'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.391099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1241/2459] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:26 2022] DELETE /identity/v3/users/5e05c9bdfc114478b5d80f077b970d17 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:01:26.580830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:26.594227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79308773-b523-4c4a-8fc1-e26464435d71 service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:26.672193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:26.672971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:26.673343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:26.673692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] PATH_INFO: `/v3/projects/002000e5606c46c29c87b0fa2fe68b3c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:26.674922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=002000e5606c46c29c87b0fa2fe68b3c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:26.677931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:26.733391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002000e5606c46c29c87b0fa2fe68b3c', 'request_id': 'req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.740423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002000e5606c46c29c87b0fa2fe68b3c', 'request_id': 'req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.748532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002000e5606c46c29c87b0fa2fe68b3c', 'request_id': 'req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.760763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] The token cache is being invalidate because project 002000e5606c46c29c87b0fa2fe68b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:26.761054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.761698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6c3d1bb4-7c69-4eca-9b08-7504730b5d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:26.763223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1219/2460] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:26 2022] DELETE /identity/v3/projects/002000e5606c46c29c87b0fa2fe68b3c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:26.769992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e5fc6-d8af-498b-925c-cf1db98881b7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:26.770297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e5fc6-d8af-498b-925c-cf1db98881b7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:26.770576 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e5fc6-d8af-498b-925c-cf1db98881b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:26.790557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-af7e5fc6-d8af-498b-925c-cf1db98881b7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:26.791753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-af7e5fc6-d8af-498b-925c-cf1db98881b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:26.855482 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1242/2461] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:26.865502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789b554-bc94-4a6e-9699-0871ba844b16 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:26.865786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789b554-bc94-4a6e-9699-0871ba844b16 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:26.866069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789b554-bc94-4a6e-9699-0871ba844b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:26.885234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-f789b554-bc94-4a6e-9699-0871ba844b16 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:26.886428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f789b554-bc94-4a6e-9699-0871ba844b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:26.895413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1220/2462] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:26.899647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e5fc6-d8af-498b-925c-cf1db98881b7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:26.909355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e5fc6-d8af-498b-925c-cf1db98881b7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:26.914822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:26.915441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:26.915730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:26.916011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:26.916752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:01:26.917184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:26.918820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:26.931102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1243/2463] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:01:26.935325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789b554-bc94-4a6e-9699-0871ba844b16 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:26.944710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789b554-bc94-4a6e-9699-0871ba844b16 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:26.949636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:26.950251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:26.950564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:26.950841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:26.951503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:26.953576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:26.981739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1221/2464] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:26 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:01:26.985913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:26.995811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14062e-c66b-48b2-82fe-273db11c2abc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:27.007747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:27.054388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:27.055010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:27.055293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:27.055567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:27.056268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:27.058220 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:27.078679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1244/2465] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:26 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:27.083243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:27.093598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dac72-5964-453d-842d-a1e6e55e5f44 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:27.099126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:27.099739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:27.100020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:27.100316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:27.101058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:27.102786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:27.114371 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1222/2466] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:27.118457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:27.129549 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184073da-e60b-44b2-85a9-ad468b95fcaf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:27.134515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a9738-0ac1-445c-8c5c-8fc9defa8f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:27.135127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9738-0ac1-445c-8c5c-8fc9defa8f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:27.135406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9738-0ac1-445c-8c5c-8fc9defa8f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:27.135678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9738-0ac1-445c-8c5c-8fc9defa8f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:27.136360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a9738-0ac1-445c-8c5c-8fc9defa8f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:27.137976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a9738-0ac1-445c-8c5c-8fc9defa8f50 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:27.149906 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1245/2467] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:27.154196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:27.165407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36ab9-302b-4138-b3ac-6e6ad9db0973 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:27.170107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:27.170780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:27.171066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:27.171341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] PATH_INFO: `/v3/projects/387f03d497884045b998a6f788991e18/users/13af20e11c9449fd807eeeb1b3ff898f/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:27.175220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:27.188873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387f03d497884045b998a6f788991e18, user_id=13af20e11c9449fd807eeeb1b3ff898f, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:27.192201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:27.205423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1223/2468] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:27 2022] PUT /identity/v3/projects/387f03d497884045b998a6f788991e18/users/13af20e11c9449fd807eeeb1b3ff898f/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:01:27.211294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c6c9e-a35f-45f3-9dd5-4c5171615bdf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:27.211582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c6c9e-a35f-45f3-9dd5-4c5171615bdf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:27.211861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c6c9e-a35f-45f3-9dd5-4c5171615bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:27.231282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c87c6c9e-a35f-45f3-9dd5-4c5171615bdf None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:27.232504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c87c6c9e-a35f-45f3-9dd5-4c5171615bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:27.292723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1246/2469] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:27.304104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:27.306493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:27.365778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7593d0-88cf-4a58-82e5-6183626c8202 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:27.369583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d95c9-4b1a-4284-8bc0-92eaab313eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:27.370318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d95c9-4b1a-4284-8bc0-92eaab313eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:27.370603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d95c9-4b1a-4284-8bc0-92eaab313eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:27.370876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d95c9-4b1a-4284-8bc0-92eaab313eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:27.382268 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d95c9-4b1a-4284-8bc0-92eaab313eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:27.384262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d95c9-4b1a-4284-8bc0-92eaab313eb9 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:27.397062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1224/2470] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:28.975592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c6c9e-a35f-45f3-9dd5-4c5171615bdf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:28.985822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c6c9e-a35f-45f3-9dd5-4c5171615bdf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:28.990372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:28.991007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:28.991317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:28.991598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:28.993554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:29.006666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:29.008099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:29.018553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1247/2471] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:01:33.117372 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ef106-9ddb-4797-8b44-6dc8e26218c9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:33.117911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ef106-9ddb-4797-8b44-6dc8e26218c9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:33.117911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ef106-9ddb-4797-8b44-6dc8e26218c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:33.183359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5e0ef106-9ddb-4797-8b44-6dc8e26218c9 None None] MFA Rules not processed for user `13af20e11c9449fd807eeeb1b3ff898f`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:33.185016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5e0ef106-9ddb-4797-8b44-6dc8e26218c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:33.309211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1225/2472] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:33.313471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:33.325216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af07088-5150-4d60-9a9c-092a9684d80e service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:33.330426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:33.331318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:33.332422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:33.333504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:33.334399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:33.336368 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:33.386009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ef106-9ddb-4797-8b44-6dc8e26218c9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:33.403266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1248/2473] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:33 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:33.408689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ef106-9ddb-4797-8b44-6dc8e26218c9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:33.518850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:33.520091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:33.520382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:33.520729 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:33.528583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:33.528583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:33.584360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1226/2474] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:33 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:01:33.588650 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:33.615092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789fc0-3d60-4572-842e-4907dfde4a61 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:33.628679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:33.629563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:33.630224 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:33.630609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:33.631426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:33.637233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:33.659001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1249/2475] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:33.664444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:33.680531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa94ba5-ca53-48c4-8572-54dd8b1e6c40 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:33.694188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:33.694973 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:33.695171 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:33.695456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] PATH_INFO: `/v3/projects/a05af5175bb843b0ab3c0e78156c518d/users/19abe8b7770d47ffa9e8af720bd8f123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:33.703732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:33.741500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05af5175bb843b0ab3c0e78156c518d, user_id=19abe8b7770d47ffa9e8af720bd8f123, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:33.744400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:33.776976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1227/2476] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:33 2022] PUT /identity/v3/projects/a05af5175bb843b0ab3c0e78156c518d/users/19abe8b7770d47ffa9e8af720bd8f123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:33.778648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:33.791420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc28538-eb48-4ca2-8943-fb43a6e54d85 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:33.887570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:33.891241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:33.896390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:33.914928 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:33.916777 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:33.917379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:33.917904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:33.919542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:33.922553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:33.944808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1250/2477] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:33.953473 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:33.971603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd0b23-9660-4fe4-96e4-0135365f5874 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:33.978565 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:33.980094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:33.980465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:33.980840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] PATH_INFO: `/v3/projects/a05af5175bb843b0ab3c0e78156c518d/users/19abe8b7770d47ffa9e8af720bd8f123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:33.983486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05af5175bb843b0ab3c0e78156c518d, user_id=19abe8b7770d47ffa9e8af720bd8f123, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:33.987000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:34.007933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf536762-33e7-44a7-ad00-2d3c8773cf73 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:34.009740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1251/2478] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:33 2022] PUT /identity/v3/projects/a05af5175bb843b0ab3c0e78156c518d/users/19abe8b7770d47ffa9e8af720bd8f123/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:34.010897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:34.012807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:34.118901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b03e74-9934-4f53-a387-f22f25deed89 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:34.122338 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:34.123350 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:34.123758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:34.124113 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:34.124394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:34.125184 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:34.125466 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:34.125745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] PATH_INFO: `/v3/users/b34e3e2e2e6049428295588959ca595c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:34.126791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:34.126992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b34e3e2e2e6049428295588959ca595c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:34.130131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:34.149686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:34.151441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:34.170037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1228/2479] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:01:34.228607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b34e3e2e2e6049428295588959ca595c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:34.240818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b34e3e2e2e6049428295588959ca595c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:34.278127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b34e3e2e2e6049428295588959ca595c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:34.287002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b34e3e2e2e6049428295588959ca595c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:34.287863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7df29129-efcb-4091-8b0c-c4852f3f07b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b34e3e2e2e6049428295588959ca595c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:34.296361 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1252/2480] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:34 2022] DELETE /identity/v3/users/b34e3e2e2e6049428295588959ca595c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:34.821137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:34.838741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb705aca-2f37-4ee2-9560-b58c4700a7bc service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:34.927571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:34.928356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:34.928794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:34.929078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] PATH_INFO: `/v3/projects/872e2c13f9b346598c261de5609fdf89` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:34.930241 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] RBAC: Authorizing `identity:delete_project(project_id=872e2c13f9b346598c261de5609fdf89)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:34.932840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:34.954776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:34.995795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872e2c13f9b346598c261de5609fdf89', 'request_id': 'req-efb9228d-5811-4097-adfa-752d21c5d5db'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:35.003407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872e2c13f9b346598c261de5609fdf89', 'request_id': 'req-efb9228d-5811-4097-adfa-752d21c5d5db'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:35.014755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872e2c13f9b346598c261de5609fdf89', 'request_id': 'req-efb9228d-5811-4097-adfa-752d21c5d5db'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:35.028305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] The token cache is being invalidate because project 872e2c13f9b346598c261de5609fdf89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:35.028660 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:35.029348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-efb9228d-5811-4097-adfa-752d21c5d5db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:35.030859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1229/2481] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:34 2022] DELETE /identity/v3/projects/872e2c13f9b346598c261de5609fdf89 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:35.037263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a720e-6266-4329-ae8e-f2f6996df9fb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.037766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a720e-6266-4329-ae8e-f2f6996df9fb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.037945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a720e-6266-4329-ae8e-f2f6996df9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.061486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ea8a720e-6266-4329-ae8e-f2f6996df9fb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:35.063073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ea8a720e-6266-4329-ae8e-f2f6996df9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:35.122240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:35.140312 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1253/2482] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:35.148261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13a1d-13e6-46e4-8f2e-93cba519cdca None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.148854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13a1d-13e6-46e4-8f2e-93cba519cdca None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.148854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a13a1d-13e6-46e4-8f2e-93cba519cdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.186484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b5a13a1d-13e6-46e4-8f2e-93cba519cdca None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:35.188325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b5a13a1d-13e6-46e4-8f2e-93cba519cdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:35.202497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1230/2483] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:35.203779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a720e-6266-4329-ae8e-f2f6996df9fb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:35.214780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a720e-6266-4329-ae8e-f2f6996df9fb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:35.220328 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:35.221946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.221946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.221946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.222852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:01:35.223320 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:35.225040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:35.241692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1254/2484] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:01:35.243442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a13a1d-13e6-46e4-8f2e-93cba519cdca None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:35.256638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a13a1d-13e6-46e4-8f2e-93cba519cdca None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:35.271824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:35.272503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.272808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.273266 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.274202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:35.276442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:35.328130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1231/2485] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:35 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:35.329306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:35.345167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f043e9-a72d-4574-9242-38329b81e1e4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:35.420088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:35.420788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.421176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.421672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.422430 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:35.424490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:35.453944 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1255/2486] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:35 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:35.458425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:35.470675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb118084-0b82-444b-8a44-580641d79a91 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:35.478348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:35.479277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.479561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.479838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.480560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:35.482196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:35.494182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1232/2487] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:01:35.498262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:35.509872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b579c64-9b46-4dc3-b649-4f26ef4db98b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:35.514569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d328cf-7ca0-46a1-8829-9797a152af9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:35.515605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d328cf-7ca0-46a1-8829-9797a152af9b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.515605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d328cf-7ca0-46a1-8829-9797a152af9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.515893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d328cf-7ca0-46a1-8829-9797a152af9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.516626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d328cf-7ca0-46a1-8829-9797a152af9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:35.518352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d328cf-7ca0-46a1-8829-9797a152af9b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:35.531092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1256/2488] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:35.535918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:35.552011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a061-d50a-4a36-9e5f-c46cdbaf7e59 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:35.557620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:35.558848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.559295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.559726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] PATH_INFO: `/v3/projects/df275d2c8a94434cbe030988b550efab/users/4081ed3f610d49fea33a0c72019fd05c/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.580465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df275d2c8a94434cbe030988b550efab, user_id=4081ed3f610d49fea33a0c72019fd05c, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:35.584243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:35.630767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1233/2489] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:35 2022] PUT /identity/v3/projects/df275d2c8a94434cbe030988b550efab/users/4081ed3f610d49fea33a0c72019fd05c/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:35.633955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4c89e-8f9b-44d3-88df-e4e994a1dc22 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:35.634246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4c89e-8f9b-44d3-88df-e4e994a1dc22 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:35.634577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4c89e-8f9b-44d3-88df-e4e994a1dc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:35.671951 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-12e4c89e-8f9b-44d3-88df-e4e994a1dc22 None None] MFA Rules not processed for user `4081ed3f610d49fea33a0c72019fd05c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:35.673795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-12e4c89e-8f9b-44d3-88df-e4e994a1dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:35.840820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1257/2490] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:35.842335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:36.041475 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e4495-a992-4f6f-85f3-88ba5437a828 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:36.047903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:36.048568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:36.048869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:36.049279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:36.049955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:36.063000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:36.122124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1234/2491] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:35 2022] POST /identity/v3/projects => generated 371 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:36.125966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4c89e-8f9b-44d3-88df-e4e994a1dc22 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:36.142851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4c89e-8f9b-44d3-88df-e4e994a1dc22 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:36.219416 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:36.238852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:36.240147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:36.240516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:36.241017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:36.241914 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:36.243935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:36.276302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1258/2492] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:36 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:01:36.279894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:36.300663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b71418-eb62-4af5-a608-07869c0a01eb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:36.308620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:36.310119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:36.311842 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:36.313522 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:36.315023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:36.320587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:36.341086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1235/2493] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:36.341808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:36.379666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448664e-8528-492b-bf62-b48b6c3b7577 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:36.397814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:36.399053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:36.399348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:36.399635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] PATH_INFO: `/v3/projects/245048694a754e27bfdcde3190bc6d5b/users/9689df00dc8e4210ba9157f05dcd6d24/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:36.445666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] RBAC: Authorizing `identity:create_grant(project_id=245048694a754e27bfdcde3190bc6d5b, user_id=9689df00dc8e4210ba9157f05dcd6d24, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:36.448372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:36.481443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1259/2494] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:36 2022] PUT /identity/v3/projects/245048694a754e27bfdcde3190bc6d5b/users/9689df00dc8e4210ba9157f05dcd6d24/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:36.485403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:36.501637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983dedf9-7340-4556-9e40-b2175dc3a968 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:36.527771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:36.617356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f6381-d86c-4faa-bb0b-06db3aa900d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:36.618260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f6381-d86c-4faa-bb0b-06db3aa900d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:36.618549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f6381-d86c-4faa-bb0b-06db3aa900d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:36.618831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f6381-d86c-4faa-bb0b-06db3aa900d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:36.619541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f6381-d86c-4faa-bb0b-06db3aa900d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:36.621202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f6381-d86c-4faa-bb0b-06db3aa900d4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:36.633541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1236/2495] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:36.637603 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:36.656018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c26f-6518-45a1-ba96-70fe13a7462d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:36.667492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:36.668190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:36.668475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:36.668782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] PATH_INFO: `/v3/projects/245048694a754e27bfdcde3190bc6d5b/users/9689df00dc8e4210ba9157f05dcd6d24/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:36.674558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=245048694a754e27bfdcde3190bc6d5b, user_id=9689df00dc8e4210ba9157f05dcd6d24, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:36.681198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:36.696932 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1260/2496] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:36 2022] PUT /identity/v3/projects/245048694a754e27bfdcde3190bc6d5b/users/9689df00dc8e4210ba9157f05dcd6d24/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:36.711490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539f13-0d76-4688-b70d-95418a6dc4c1 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:36.712272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539f13-0d76-4688-b70d-95418a6dc4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:36.713103 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539f13-0d76-4688-b70d-95418a6dc4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:36.788568 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-1a539f13-0d76-4688-b70d-95418a6dc4c1 None None] MFA Rules not processed for user `9689df00dc8e4210ba9157f05dcd6d24`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:36.790020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1a539f13-0d76-4688-b70d-95418a6dc4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:36.921204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1237/2497] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:36.933692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:36.935492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:37.067434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb573b1-48ce-4f9c-bbfa-8ee2d329b640 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:37.079783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d744ef-ba9c-4f0a-8994-63543c83907c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:37.080442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d744ef-ba9c-4f0a-8994-63543c83907c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:37.081280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d744ef-ba9c-4f0a-8994-63543c83907c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:37.081883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d744ef-ba9c-4f0a-8994-63543c83907c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:37.112909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d744ef-ba9c-4f0a-8994-63543c83907c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:37.114678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d744ef-ba9c-4f0a-8994-63543c83907c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:37.134505 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1261/2498] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:37.995579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539f13-0d76-4688-b70d-95418a6dc4c1 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:38.079539 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539f13-0d76-4688-b70d-95418a6dc4c1 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:38.091075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:38.092045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:38.093172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:38.093952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] PATH_INFO: `/v3/users/4081ed3f610d49fea33a0c72019fd05c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:38.096877 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4081ed3f610d49fea33a0c72019fd05c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:38.101643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:38.173022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:38.182914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4081ed3f610d49fea33a0c72019fd05c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:38.183986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4081ed3f610d49fea33a0c72019fd05c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:38.213414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4081ed3f610d49fea33a0c72019fd05c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:38.233644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4081ed3f610d49fea33a0c72019fd05c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:38.234629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4081ed3f610d49fea33a0c72019fd05c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:38.286070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1238/2499] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:37 2022] DELETE /identity/v3/users/4081ed3f610d49fea33a0c72019fd05c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:01:38.290633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:38.291503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:38.292082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:38.340853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:38.342261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:38.563085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1262/2500] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:38.582976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:38.584975 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:38.721283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a220f9-592e-4245-a480-582789f52e8a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:38.731309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:38.731309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:38.731309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:38.731309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:38.742554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:38.744787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:38.757490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1239/2501] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:39.919135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:39.920838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:39.954113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dee9f-b771-45fd-b403-caf7af374b68 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:39.980632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:39.981423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:39.981715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:39.981996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] PATH_INFO: `/v3/projects/df275d2c8a94434cbe030988b550efab` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:39.983363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df275d2c8a94434cbe030988b550efab)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:39.985316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:40.069797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:40.086199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df275d2c8a94434cbe030988b550efab', 'request_id': 'req-372ed7bd-f1d4-4585-9057-834c3fb41c35'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.106620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df275d2c8a94434cbe030988b550efab', 'request_id': 'req-372ed7bd-f1d4-4585-9057-834c3fb41c35'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.132791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df275d2c8a94434cbe030988b550efab', 'request_id': 'req-372ed7bd-f1d4-4585-9057-834c3fb41c35'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.168217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] The token cache is being invalidate because project df275d2c8a94434cbe030988b550efab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:40.173743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.174868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.177547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1263/2502] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:39 2022] DELETE /identity/v3/projects/df275d2c8a94434cbe030988b550efab => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:40.181951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:40.195693 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8b6cc-d74e-41a0-a454-b3756ffc810e service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:40.332689 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:40.333873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:40.334334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:40.334780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] PATH_INFO: `/v3/users/9689df00dc8e4210ba9157f05dcd6d24` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:40.338344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9689df00dc8e4210ba9157f05dcd6d24)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:40.340738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:40.469695 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9689df00dc8e4210ba9157f05dcd6d24'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.470399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9689df00dc8e4210ba9157f05dcd6d24'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.486219 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:40.490124 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9689df00dc8e4210ba9157f05dcd6d24'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.503387 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9689df00dc8e4210ba9157f05dcd6d24'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.505309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-febdd699-4db1-43d7-8c72-d2b11ba51c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9689df00dc8e4210ba9157f05dcd6d24'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:40.536511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1240/2503] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:40 2022] DELETE /identity/v3/users/9689df00dc8e4210ba9157f05dcd6d24 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:41.114093 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.116117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:41.136876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ed7bd-f1d4-4585-9057-834c3fb41c35 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.219143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.220108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.220468 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.220858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] PATH_INFO: `/v3/projects/245048694a754e27bfdcde3190bc6d5b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.223245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245048694a754e27bfdcde3190bc6d5b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.225517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.250011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.251246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.251763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.315886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] MFA Rules not processed for user `19abe8b7770d47ffa9e8af720bd8f123`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:41.317184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:41.321457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245048694a754e27bfdcde3190bc6d5b', 'request_id': 'req-653a64b7-50ce-4dda-aa76-34b19d24a360'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:41.337566 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245048694a754e27bfdcde3190bc6d5b', 'request_id': 'req-653a64b7-50ce-4dda-aa76-34b19d24a360'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:41.350006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245048694a754e27bfdcde3190bc6d5b', 'request_id': 'req-653a64b7-50ce-4dda-aa76-34b19d24a360'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:41.375371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] The token cache is being invalidate because project 245048694a754e27bfdcde3190bc6d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:41.375812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:41.378606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-653a64b7-50ce-4dda-aa76-34b19d24a360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:41.381337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1264/2504] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:41 2022] DELETE /identity/v3/projects/245048694a754e27bfdcde3190bc6d5b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:41.395559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbdff1-9cad-40bc-8dbd-8fbd9f8ff83d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.395852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbdff1-9cad-40bc-8dbd-8fbd9f8ff83d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.396134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbdff1-9cad-40bc-8dbd-8fbd9f8ff83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.427461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e2dbdff1-9cad-40bc-8dbd-8fbd9f8ff83d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:41.428877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e2dbdff1-9cad-40bc-8dbd-8fbd9f8ff83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:41.462736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1241/2505] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:41.472831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.476285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:41.489096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:41.507821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1265/2506] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:41.517419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5022020-c81f-4c37-946c-800c1d3f53a8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.518112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5022020-c81f-4c37-946c-800c1d3f53a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.518237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5022020-c81f-4c37-946c-800c1d3f53a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.549741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f5022020-c81f-4c37-946c-800c1d3f53a8 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:41.551499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f5022020-c81f-4c37-946c-800c1d3f53a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:41.567634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1266/2507] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:41.571509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5022020-c81f-4c37-946c-800c1d3f53a8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.582491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5022020-c81f-4c37-946c-800c1d3f53a8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.589961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.590986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.591341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.591686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.592833 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:01:41.593355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.595957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.615345 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665d635-1cc3-4549-a620-13155ce04c65 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.619543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4b8f7545934151aafa986da871b944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.620239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.620576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.620872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.622738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:41.629897 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1267/2508] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 07:01:41.630682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.647937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c522746-75f8-4251-a8bb-5dd6553df72e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.654971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.655619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.655998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.656335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.657371 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.659358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.700282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1268/2509] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:41 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:41.705117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.717443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.719098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.729273 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:41.738893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1242/2510] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:01:41.801157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba45773-23a9-4b55-adbe-fea89e919fce admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.806939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83e24d-6460-4770-9590-20c0ffc7ad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.807577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e24d-6460-4770-9590-20c0ffc7ad46 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.807860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e24d-6460-4770-9590-20c0ffc7ad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.808140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e24d-6460-4770-9590-20c0ffc7ad46 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.808848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83e24d-6460-4770-9590-20c0ffc7ad46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.811087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83e24d-6460-4770-9590-20c0ffc7ad46 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.846701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1269/2511] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:41 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:41.851796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.867156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3cea5-0280-48bb-a606-f35463957513 service glance] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.872495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.873193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.873479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.874428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.874553 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.876117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.888582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1243/2512] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:41.889512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.889810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.890101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.891460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.903629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de4d8-3135-4f20-a762-118198ecf16b admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.911876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.912122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.915717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.915717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.915717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.915717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.926459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1244/2513] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:41.930905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:41.944996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45c474-e589-4eb8-9ed3-b9cf2cc5bd45 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:41.950840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad737f-8783-48ed-8248-ef109ee9a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:41.951536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad737f-8783-48ed-8248-ef109ee9a93f admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:41.951818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad737f-8783-48ed-8248-ef109ee9a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:41.952430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad737f-8783-48ed-8248-ef109ee9a93f admin admin] PATH_INFO: `/v3/projects/eb3ea48818d3477abaebd59c9ae6dc2b/users/d418298f75d84107b337b96e17746959/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:41.961384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:41.963216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:41.968929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad737f-8783-48ed-8248-ef109ee9a93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3ea48818d3477abaebd59c9ae6dc2b, user_id=d418298f75d84107b337b96e17746959, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:41.971579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad737f-8783-48ed-8248-ef109ee9a93f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:41.990431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1245/2514] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:41 2022] PUT /identity/v3/projects/eb3ea48818d3477abaebd59c9ae6dc2b/users/d418298f75d84107b337b96e17746959/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:01:42.001769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:42.002061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:42.002343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:42.043039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] MFA Rules not processed for user `d418298f75d84107b337b96e17746959`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:42.044378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:42.054171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1270/2515] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 07:01:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:42.108717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:42.110604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:42.119988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1246/2516] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:42.126580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:42.135091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:42.136678 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:42.204779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dfcac-f6fd-4926-a8be-74ea56133669 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:42.209593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:42.210175 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:42.210463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:42.210741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:42.224946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:42.227238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:42.232817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25d2a3-fa27-42a7-88e8-bce441ddd373 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:42.252258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1247/2517] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:42.264033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:42.316662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:42.316970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dd2fb-62c7-429b-b814-88fdf6ef3259 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:42.317370 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dd2fb-62c7-429b-b814-88fdf6ef3259 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:42.317653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dd2fb-62c7-429b-b814-88fdf6ef3259 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:42.317921 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dd2fb-62c7-429b-b814-88fdf6ef3259 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:42.343548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a045b5-5cde-4ce2-94ff-792941cad9fa service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:42.348557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:42.349221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:42.349508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:42.349785 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] PATH_INFO: `/v3/users/d418298f75d84107b337b96e17746959` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:42.351023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d418298f75d84107b337b96e17746959)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:42.352985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:42.435318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd418298f75d84107b337b96e17746959'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:42.437110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd418298f75d84107b337b96e17746959'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:42.440031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dd2fb-62c7-429b-b814-88fdf6ef3259 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:42.442283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dd2fb-62c7-429b-b814-88fdf6ef3259 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:42.452116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1271/2518] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 07:01:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 07:01:42.466128 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd418298f75d84107b337b96e17746959'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:42.476386 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd418298f75d84107b337b96e17746959'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:42.478112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd418298f75d84107b337b96e17746959'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:42.538089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1248/2519] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:42 2022] DELETE /identity/v3/users/d418298f75d84107b337b96e17746959 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:42.542327 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39dfc-d1c1-4c6c-ae83-0b71b22387c2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:42.542625 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39dfc-d1c1-4c6c-ae83-0b71b22387c2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:42.542927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39dfc-d1c1-4c6c-ae83-0b71b22387c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:42.575044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4fe39dfc-d1c1-4c6c-ae83-0b71b22387c2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:42.578347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4fe39dfc-d1c1-4c6c-ae83-0b71b22387c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:42.654379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:42.729041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1272/2520] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:42.741345 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:42.743328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:42.777405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:42.777876 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:42.778022 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:42.804172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] MFA Rules not processed for user `12a751b41fd546ac9017c67d6de4ebdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:42.805585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:42.872005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478b2dc-b8ed-4bea-8668-f89e1f7afbee admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:42.877486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:42.878838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:42.879276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:42.879561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:42.884131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1273/2521] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:42.891223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:42.893293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:42.906229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1249/2522] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:43.323512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:43.325081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:43.355696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc051a2-1ce0-4a76-8fc4-f6139c024602 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:43.362298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:43.363542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:43.363826 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:43.364105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] PATH_INFO: `/v3/projects/eb3ea48818d3477abaebd59c9ae6dc2b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:43.385001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:43.386402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:43.395128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3ea48818d3477abaebd59c9ae6dc2b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:43.397159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:43.410167 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def86da-9c4c-4986-a8d0-77a08c1f1001 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:43.416381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec1524-841b-429e-826f-a65a916d36cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:43.417444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec1524-841b-429e-826f-a65a916d36cf service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:43.417832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec1524-841b-429e-826f-a65a916d36cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:43.419806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec1524-841b-429e-826f-a65a916d36cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:43.424353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-fcec1524-841b-429e-826f-a65a916d36cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:43.437997 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec1524-841b-429e-826f-a65a916d36cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:43.439759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec1524-841b-429e-826f-a65a916d36cf service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:43.451332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1250/2523] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:43.475041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3ea48818d3477abaebd59c9ae6dc2b', 'request_id': 'req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:43.486433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3ea48818d3477abaebd59c9ae6dc2b', 'request_id': 'req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:43.496136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3ea48818d3477abaebd59c9ae6dc2b', 'request_id': 'req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:43.515039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] The token cache is being invalidate because project eb3ea48818d3477abaebd59c9ae6dc2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:43.515461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:43.517744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16bb6d28-298f-4f70-9ba0-e6c0b1cb2992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:43.520991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1274/2524] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:43 2022] DELETE /identity/v3/projects/eb3ea48818d3477abaebd59c9ae6dc2b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:43.526173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fce79c-4f59-4600-8dd3-3ddfde764816 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:43.526480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fce79c-4f59-4600-8dd3-3ddfde764816 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:43.526766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fce79c-4f59-4600-8dd3-3ddfde764816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:43.576663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-41fce79c-4f59-4600-8dd3-3ddfde764816 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:43.578240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-41fce79c-4f59-4600-8dd3-3ddfde764816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:43.679438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:43.709230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1251/2525] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:43.717605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b5406-8fd8-464f-82c0-115a4fe9c0ce None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:43.717973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b5406-8fd8-464f-82c0-115a4fe9c0ce None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:43.718901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b5406-8fd8-464f-82c0-115a4fe9c0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:43.749211 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1f4b5406-8fd8-464f-82c0-115a4fe9c0ce None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:43.750382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1f4b5406-8fd8-464f-82c0-115a4fe9c0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:43.789748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1275/2526] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:43.793280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fce79c-4f59-4600-8dd3-3ddfde764816 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:43.804446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fce79c-4f59-4600-8dd3-3ddfde764816 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:43.811288 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:43.812025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:43.812311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:43.812826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:43.813398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:01:43.813704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:43.815413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:43.831218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1252/2527] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:01:43.837392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b5406-8fd8-464f-82c0-115a4fe9c0ce None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:43.848036 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b5406-8fd8-464f-82c0-115a4fe9c0ce None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:43.853999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:43.854770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:43.855059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:43.855478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:43.856156 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:43.858140 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:43.920911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1276/2528] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:43 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:43.927639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:43.973443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabcb39-be7e-40b2-9968-322d25e5ea51 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:44.052562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:44.057879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:44.106368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:44.107008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.107291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.107576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.108243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:44.111818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:44.147873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:44.156099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1253/2529] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:43 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:44.158207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:44.173861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd564b9-7fe4-4cf0-8b69-d475d83c8d76 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:44.181786 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:44.182551 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.182844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.183243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.183936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:44.185914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:44.203220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1254/2530] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:01:44.207922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:44.224630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b17f47-bff7-4cd2-ac11-207db12af2f7 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:44.234647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:44.236401 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.237010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.238027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.239056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:44.242861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdc4f1-f59e-425f-87f4-b9fec7e001c0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:44.244330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:44.260913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6b404-4646-4efe-a646-7e1120abae28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:44.261633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6b404-4646-4efe-a646-7e1120abae28 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.262109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1255/2531] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:44.262706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6b404-4646-4efe-a646-7e1120abae28 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.262706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6b404-4646-4efe-a646-7e1120abae28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.265655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:44.268251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eed6b404-4646-4efe-a646-7e1120abae28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:44.277945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fafbe2-6acb-44db-9461-e9f58c0a4775 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:44.283654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:44.284459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.285041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.285403 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] PATH_INFO: `/v3/projects/5da48cf5b7134a6095029ac01cbe0a09/users/395030c3dbd74f328970d1fe1c039159/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.293355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6b404-4646-4efe-a646-7e1120abae28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:44.294981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6b404-4646-4efe-a646-7e1120abae28 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:44.324171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1277/2532] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:44.328231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da48cf5b7134a6095029ac01cbe0a09, user_id=395030c3dbd74f328970d1fe1c039159, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:44.331867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:44.358555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1256/2533] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:44 2022] PUT /identity/v3/projects/5da48cf5b7134a6095029ac01cbe0a09/users/395030c3dbd74f328970d1fe1c039159/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:01:44.370861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b15ee-3d62-4efb-8948-17571505040a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.371353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b15ee-3d62-4efb-8948-17571505040a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.371812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b15ee-3d62-4efb-8948-17571505040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.440366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-175b15ee-3d62-4efb-8948-17571505040a None None] MFA Rules not processed for user `395030c3dbd74f328970d1fe1c039159`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:44.441744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-175b15ee-3d62-4efb-8948-17571505040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:44.613350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1278/2534] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:44.630579 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:44.634179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:44.721145 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8f3c1-b5f4-4e98-9fd5-68482d5b12a2 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:44.740894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724191c-5734-40e6-b819-da7b331d28e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:44.740894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724191c-5734-40e6-b819-da7b331d28e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.740894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724191c-5734-40e6-b819-da7b331d28e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.740894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724191c-5734-40e6-b819-da7b331d28e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.765242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724191c-5734-40e6-b819-da7b331d28e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:44.766932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724191c-5734-40e6-b819-da7b331d28e0 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:44.778655 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1257/2535] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:44.809904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b15ee-3d62-4efb-8948-17571505040a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:44.913046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b15ee-3d62-4efb-8948-17571505040a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:44.918700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:44.919764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:44.920054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:44.920332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] PATH_INFO: `/v3/users/395030c3dbd74f328970d1fe1c039159` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:44.921958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=395030c3dbd74f328970d1fe1c039159)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:44.923899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:45.000034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395030c3dbd74f328970d1fe1c039159'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:45.020435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:45.023901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395030c3dbd74f328970d1fe1c039159'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:45.052745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395030c3dbd74f328970d1fe1c039159'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:45.072167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395030c3dbd74f328970d1fe1c039159'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:45.073558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395030c3dbd74f328970d1fe1c039159'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:45.090412 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1279/2536] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:44 2022] DELETE /identity/v3/users/395030c3dbd74f328970d1fe1c039159 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:45.094747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:45.095044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:45.095358 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:45.123626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:45.127443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:45.169450 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:45.232614 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1258/2537] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:45.246911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:45.248454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:45.375477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a06e32-2453-4945-86ee-78f88345bd2e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:45.382814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6b29f-8e13-442e-82a5-2a3c676e065f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:45.383780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6b29f-8e13-442e-82a5-2a3c676e065f service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:45.383906 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6b29f-8e13-442e-82a5-2a3c676e065f service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:45.384262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6b29f-8e13-442e-82a5-2a3c676e065f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:45.401062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6b29f-8e13-442e-82a5-2a3c676e065f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:45.402635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6b29f-8e13-442e-82a5-2a3c676e065f service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:45.414457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1280/2538] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:45.907570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:45.909107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:45.953732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4dd7f-5fa2-43d8-b7d9-5e55866954a9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:45.969444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:45.970117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:45.970409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:45.970683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] PATH_INFO: `/v3/projects/5da48cf5b7134a6095029ac01cbe0a09` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:45.972856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5da48cf5b7134a6095029ac01cbe0a09)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:45.975112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:46.062768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5da48cf5b7134a6095029ac01cbe0a09', 'request_id': 'req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:46.074085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5da48cf5b7134a6095029ac01cbe0a09', 'request_id': 'req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:46.091182 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5da48cf5b7134a6095029ac01cbe0a09', 'request_id': 'req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:46.122259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] The token cache is being invalidate because project 5da48cf5b7134a6095029ac01cbe0a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:46.122993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:46.123750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-11e44d8c-ae02-4b5d-9a06-39fd727f71e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:46.126129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1259/2539] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:45 2022] DELETE /identity/v3/projects/5da48cf5b7134a6095029ac01cbe0a09 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:46.131165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe90ae4-6b6d-45d9-b47e-26025d5ebc39 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:46.131467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe90ae4-6b6d-45d9-b47e-26025d5ebc39 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:46.131799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe90ae4-6b6d-45d9-b47e-26025d5ebc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:46.163699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-abe90ae4-6b6d-45d9-b47e-26025d5ebc39 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:46.165613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-abe90ae4-6b6d-45d9-b47e-26025d5ebc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:46.330590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1281/2540] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:46.333796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940dd5e-c653-4a83-b117-1e37d6f7196b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:46.334408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940dd5e-c653-4a83-b117-1e37d6f7196b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:46.334858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940dd5e-c653-4a83-b117-1e37d6f7196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:46.393247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-e940dd5e-c653-4a83-b117-1e37d6f7196b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:46.395782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e940dd5e-c653-4a83-b117-1e37d6f7196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:46.421203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1260/2541] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:46.425426 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe90ae4-6b6d-45d9-b47e-26025d5ebc39 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:46.446486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe90ae4-6b6d-45d9-b47e-26025d5ebc39 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:46.457016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:46.457838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:46.458113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:46.458397 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:46.459097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:01:46.460496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:46.461065 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:46.493960 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1282/2542] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:01:46.495129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940dd5e-c653-4a83-b117-1e37d6f7196b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:46.505723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940dd5e-c653-4a83-b117-1e37d6f7196b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:46.530503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:46.531148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:46.531429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:46.531697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:46.532359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:46.534594 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:46.598452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1261/2543] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:46 2022] POST /identity/v3/projects => generated 359 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:01:46.603068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:46.625926 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9852-689b-4bff-9d19-8a5c64641f8e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:46.802757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:46.803715 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:46.804226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:46.804741 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:46.805626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:46.807807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:46.840396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:46.871805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1283/2544] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:46 2022] POST /identity/v3/users => generated 280 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:46.873970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:46.889793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b352644-8c88-4fa3-b739-723d6ed5aa15 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:46.897080 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:46.898155 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:46.898651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:46.899169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:46.900077 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:46.902003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:46.914549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1262/2545] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:01:46.918621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:46.931449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb2aef-8a71-4a1c-bf3a-e5993bc6f28a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:46.938249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da37931-499f-4367-8c71-6786bd2bb96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:46.939251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37931-499f-4367-8c71-6786bd2bb96a admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:46.939540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37931-499f-4367-8c71-6786bd2bb96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:46.939834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37931-499f-4367-8c71-6786bd2bb96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:46.940655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da37931-499f-4367-8c71-6786bd2bb96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:46.942990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da37931-499f-4367-8c71-6786bd2bb96a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:46.966019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1284/2546] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:46.970795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:46.996990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55cee8-890e-4b43-8370-299cb4a8dce5 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:47.008332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:47.009510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:47.009999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:47.010451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] PATH_INFO: `/v3/projects/a658fae206cb46a592907104181a30d6/users/07a41a065eca498180dee757791da7c3/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:47.053972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a658fae206cb46a592907104181a30d6, user_id=07a41a065eca498180dee757791da7c3, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:47.057071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:47.089906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1263/2547] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:46 2022] PUT /identity/v3/projects/a658fae206cb46a592907104181a30d6/users/07a41a065eca498180dee757791da7c3/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:47.103101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef75cb-a755-4a59-9415-f7f783d63566 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:47.103653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef75cb-a755-4a59-9415-f7f783d63566 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:47.104150 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef75cb-a755-4a59-9415-f7f783d63566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:47.219345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-41ef75cb-a755-4a59-9415-f7f783d63566 None None] MFA Rules not processed for user `07a41a065eca498180dee757791da7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:47.220785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-41ef75cb-a755-4a59-9415-f7f783d63566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:47.365524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1285/2548] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:47 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:47.417955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:47.419921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:47.512683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:47.521913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74084bf4-7538-40ae-b6fe-f44963fc4d24 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:47.527369 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c12fa-ef99-4d7f-8157-78e97f1b0e28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:47.528111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c12fa-ef99-4d7f-8157-78e97f1b0e28 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:47.528464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c12fa-ef99-4d7f-8157-78e97f1b0e28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:47.530251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c12fa-ef99-4d7f-8157-78e97f1b0e28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:47.542959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c12fa-ef99-4d7f-8157-78e97f1b0e28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:47.544925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c12fa-ef99-4d7f-8157-78e97f1b0e28 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:47.560261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1264/2549] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:47 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:47.577886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef75cb-a755-4a59-9415-f7f783d63566 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:47.679735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef75cb-a755-4a59-9415-f7f783d63566 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:47.691453 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:47.692135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:47.692420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:47.692725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] PATH_INFO: `/v3/users/07a41a065eca498180dee757791da7c3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:47.694049 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a41a065eca498180dee757791da7c3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:47.695989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:47.790040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a41a065eca498180dee757791da7c3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:47.798997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a41a065eca498180dee757791da7c3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:47.813821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:47.816377 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a41a065eca498180dee757791da7c3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:47.826850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a41a065eca498180dee757791da7c3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:47.828734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a41a065eca498180dee757791da7c3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:47.845035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1286/2550] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:47 2022] DELETE /identity/v3/users/07a41a065eca498180dee757791da7c3 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:47.846125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:47.846354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:47.846640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:47.881872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:47.883270 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:47.967317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1265/2551] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:47.982516 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:47.984165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:48.068055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae79fe7-4245-4229-9e3e-b3d6554b1034 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:48.072108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c3db8-f8dd-465e-9653-ff9dcfcd69c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:48.073081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3db8-f8dd-465e-9653-ff9dcfcd69c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:48.073227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3db8-f8dd-465e-9653-ff9dcfcd69c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:48.073404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3db8-f8dd-465e-9653-ff9dcfcd69c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:48.095083 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c3db8-f8dd-465e-9653-ff9dcfcd69c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:48.098692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c3db8-f8dd-465e-9653-ff9dcfcd69c0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:48.111352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1287/2552] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:48.551883 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:48.553468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:48.632409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781baa-f00e-465c-91b8-ef76dc460aa9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:48.639301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:48.639966 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:48.640247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:48.640521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] PATH_INFO: `/v3/projects/a658fae206cb46a592907104181a30d6` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:48.641846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a658fae206cb46a592907104181a30d6)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:48.645276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:48.711652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a658fae206cb46a592907104181a30d6', 'request_id': 'req-bae9545a-0375-4ab8-b25b-898191824fd3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:48.720471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a658fae206cb46a592907104181a30d6', 'request_id': 'req-bae9545a-0375-4ab8-b25b-898191824fd3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:48.738773 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a658fae206cb46a592907104181a30d6', 'request_id': 'req-bae9545a-0375-4ab8-b25b-898191824fd3'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:48.753681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] The token cache is being invalidate because project a658fae206cb46a592907104181a30d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:48.753974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:48.754667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bae9545a-0375-4ab8-b25b-898191824fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:48.758894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1266/2553] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:48 2022] DELETE /identity/v3/projects/a658fae206cb46a592907104181a30d6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:48.765305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b02b6-e462-4039-84c4-0f0a83d67e96 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:48.766669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b02b6-e462-4039-84c4-0f0a83d67e96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:48.767427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b02b6-e462-4039-84c4-0f0a83d67e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:48.802147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-860b02b6-e462-4039-84c4-0f0a83d67e96 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:48.803372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-860b02b6-e462-4039-84c4-0f0a83d67e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:48.898582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f3c27-3261-446e-a819-9b9c9959404f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:48.898582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f3c27-3261-446e-a819-9b9c9959404f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:48.898582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f3c27-3261-446e-a819-9b9c9959404f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:48.898992 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1288/2554] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:48.935601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3e0f3c27-3261-446e-a819-9b9c9959404f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:48.937792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3e0f3c27-3261-446e-a819-9b9c9959404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:48.948607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1267/2555] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:48.952620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b02b6-e462-4039-84c4-0f0a83d67e96 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:48.969087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b02b6-e462-4039-84c4-0f0a83d67e96 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:48.975439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:48.976074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:48.976416 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:48.976732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:48.977462 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:01:48.977769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:48.980231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:48.997303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f3c27-3261-446e-a819-9b9c9959404f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:48.998484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1289/2556] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:01:49.012152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f3c27-3261-446e-a819-9b9c9959404f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.021999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.023873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.024178 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.024459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.025648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.030078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.077964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1268/2557] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:48 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:01:49.079456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.090857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc983ef6-e67b-4cc1-9bf9-b613c024755f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.159750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.160568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.161071 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.161354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.162066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.164007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.195060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1290/2558] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:49 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:49.199591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.217635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c3121-73ac-487a-b311-060369161248 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.225582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.226230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.226517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.226869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.227613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.229264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.242664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1269/2559] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:49.247034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.258360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b2e5-2a39-451a-906c-f5ad05cb29d1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.266948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1959bb8-d54a-45d2-b560-20cca2582d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.267598 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959bb8-d54a-45d2-b560-20cca2582d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.268038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959bb8-d54a-45d2-b560-20cca2582d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.268323 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959bb8-d54a-45d2-b560-20cca2582d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.269968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1959bb8-d54a-45d2-b560-20cca2582d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.271611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1959bb8-d54a-45d2-b560-20cca2582d9a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.274284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:49.288200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1291/2560] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:49.289299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.300074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715dca1-6ef4-4d04-948b-daa36bd08146 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.308000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.308742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.309032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.309316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] PATH_INFO: `/v3/projects/3191da1d6aa84528b76f8be8c7923c66/users/b49a46ec0ca54316a76632f7adabf55f/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.328996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3191da1d6aa84528b76f8be8c7923c66, user_id=b49a46ec0ca54316a76632f7adabf55f, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.332276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.358073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1270/2561] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:49 2022] PUT /identity/v3/projects/3191da1d6aa84528b76f8be8c7923c66/users/b49a46ec0ca54316a76632f7adabf55f/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:49.360357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5f679-0d55-4b04-abc4-5cc1d4f16c52 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.360909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5f679-0d55-4b04-abc4-5cc1d4f16c52 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.361908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5f679-0d55-4b04-abc4-5cc1d4f16c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.393219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0ba5f679-0d55-4b04-abc4-5cc1d4f16c52 None None] MFA Rules not processed for user `b49a46ec0ca54316a76632f7adabf55f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:49.395317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0ba5f679-0d55-4b04-abc4-5cc1d4f16c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:49.487865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1292/2562] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:49.494538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.506682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:49.576203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8502f32-3821-418e-be53-520c063bb74e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.583702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.584686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.584977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.585253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.585923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.588119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.621279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1271/2563] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:49 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:49.625486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5f679-0d55-4b04-abc4-5cc1d4f16c52 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.639693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5f679-0d55-4b04-abc4-5cc1d4f16c52 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.756127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.756983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.757374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.757752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.758542 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.760599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.794640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1293/2564] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:49 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:49.798681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.810399 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e21cb1-d8c3-415c-99eb-f99ff04958bf admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.817258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.818031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.818830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.818830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.819524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.823694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.843827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.844618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1272/2565] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:49.859717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06af89e-c59f-4984-a4e5-73ea5373ce27 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.866177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42aeb8-3a4d-4731-b48a-239b7ddb69bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.867119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42aeb8-3a4d-4731-b48a-239b7ddb69bd admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.867586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42aeb8-3a4d-4731-b48a-239b7ddb69bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.868028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42aeb8-3a4d-4731-b48a-239b7ddb69bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.868938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42aeb8-3a4d-4731-b48a-239b7ddb69bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.871107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42aeb8-3a4d-4731-b48a-239b7ddb69bd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.888739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1294/2566] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:01:49.893456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:49.904651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b48b0-5871-49d0-848d-d826da9760f3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:49.921742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:49.922448 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.922727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.923004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] PATH_INFO: `/v3/projects/0af1d349a6dd4116bee3821f09b445a5/users/26c2c97354bc43dbb44f5a0dfcf1cf84/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:49.944426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af1d349a6dd4116bee3821f09b445a5, user_id=26c2c97354bc43dbb44f5a0dfcf1cf84, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:49.948907 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:49.967748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1273/2567] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:49 2022] PUT /identity/v3/projects/0af1d349a6dd4116bee3821f09b445a5/users/26c2c97354bc43dbb44f5a0dfcf1cf84/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:01:49.978425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3928dee-2d28-49c7-aec4-c3244e9a6186 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:49.978776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3928dee-2d28-49c7-aec4-c3244e9a6186 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:49.979138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3928dee-2d28-49c7-aec4-c3244e9a6186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.027162 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d3928dee-2d28-49c7-aec4-c3244e9a6186 None None] MFA Rules not processed for user `26c2c97354bc43dbb44f5a0dfcf1cf84`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:50.039838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d3928dee-2d28-49c7-aec4-c3244e9a6186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:50.135446 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:50.158112 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1295/2568] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:50.163035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:50.249631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4652c-8347-40ba-8bcb-f63d158a15d3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:50.257276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:50.258406 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:50.258705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:50.258989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.259671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:50.261708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:50.316703 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1274/2569] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:01:50 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:50.320489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3928dee-2d28-49c7-aec4-c3244e9a6186 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:50.334431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3928dee-2d28-49c7-aec4-c3244e9a6186 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:50.420302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:50.420975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:50.421261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:50.421537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.422329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:50.425638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:50.459524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1296/2570] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:01:50 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:01:50.460846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:50.471814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64318077-acfa-4bd0-8f9a-7169a7449d8e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:50.479034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:50.479764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:50.480112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:50.480460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.481253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:50.483509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:50.498717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1275/2571] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:50.500034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:50.513360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026702f4-a44a-425b-abee-3ed6b6439c52 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:50.522294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:50.523143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:50.523526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:50.523883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] PATH_INFO: `/v3/projects/df7d0786bbe74b5d98b70e39848f0e60/users/05e4a8203569439888686fa087edd7fd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.544445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7d0786bbe74b5d98b70e39848f0e60, user_id=05e4a8203569439888686fa087edd7fd, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:50.558160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:50.575123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1297/2572] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:50 2022] PUT /identity/v3/projects/df7d0786bbe74b5d98b70e39848f0e60/users/05e4a8203569439888686fa087edd7fd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:50.579188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:50.593888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c7e15-68f1-4b33-931e-f4837fa46b8f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:50.631580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:50.664217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbaeb6-36b1-4afd-ab29-7b99450fcc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:50.666705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbaeb6-36b1-4afd-ab29-7b99450fcc60 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:50.667201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbaeb6-36b1-4afd-ab29-7b99450fcc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:50.667489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbaeb6-36b1-4afd-ab29-7b99450fcc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.668292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbaeb6-36b1-4afd-ab29-7b99450fcc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:50.671891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbaeb6-36b1-4afd-ab29-7b99450fcc60 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:50.684172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1276/2573] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:01:50.688291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:50.702251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ebda2-1bbb-4d36-9055-0c26f5d63615 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:50.707837 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:50.708660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:50.709044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:50.709443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] PATH_INFO: `/v3/projects/df7d0786bbe74b5d98b70e39848f0e60/users/05e4a8203569439888686fa087edd7fd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.711507 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7d0786bbe74b5d98b70e39848f0e60, user_id=05e4a8203569439888686fa087edd7fd, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:50.714570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:50.741114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1298/2574] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:01:50 2022] PUT /identity/v3/projects/df7d0786bbe74b5d98b70e39848f0e60/users/05e4a8203569439888686fa087edd7fd/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:01:50.752459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ad70a-f79f-457b-bf7d-936862420b18 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:50.752722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ad70a-f79f-457b-bf7d-936862420b18 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:50.753571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ad70a-f79f-457b-bf7d-936862420b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:50.787780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-426ad70a-f79f-457b-bf7d-936862420b18 None None] MFA Rules not processed for user `05e4a8203569439888686fa087edd7fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:50.789327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-426ad70a-f79f-457b-bf7d-936862420b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:50.895139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1277/2575] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:50.903099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:50.905052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:51.001299 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c06889-0fca-4f3a-be04-35cb3dcbd7ee admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:51.005606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:51.006182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:51.006469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:51.006745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:51.069902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:51.074240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:51.103195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1299/2576] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:52.754511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ad70a-f79f-457b-bf7d-936862420b18 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:52.766477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ad70a-f79f-457b-bf7d-936862420b18 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:52.773947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:52.774596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:52.775483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:52.775483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:52.855631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:52.857317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:52.868828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1278/2577] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:52.937449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:52.938950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:52.944226 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:53.087618 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5b75-6b2d-4dfd-abcf-7d9a195d5279 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:53.096257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d1dd0-0fee-4d9a-9a32-10901f1cedce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:53.097735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1dd0-0fee-4d9a-9a32-10901f1cedce service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:53.098137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1dd0-0fee-4d9a-9a32-10901f1cedce service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:53.098445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1dd0-0fee-4d9a-9a32-10901f1cedce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:53.100369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-594d1dd0-0fee-4d9a-9a32-10901f1cedce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:53.115935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d1dd0-0fee-4d9a-9a32-10901f1cedce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:53.118028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d1dd0-0fee-4d9a-9a32-10901f1cedce service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:53.131827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1300/2578] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:53.619413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:53.631211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5da6c-3bd5-4104-9207-dd86a9c00993 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:53.635454 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:53.636101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:53.636384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:53.636701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:53.650096 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:53.651662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:53.664779 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1279/2579] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:54.746666 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a334-6c4c-478e-ba59-24cc1f2878a5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:54.747135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a334-6c4c-478e-ba59-24cc1f2878a5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:54.747255 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a334-6c4c-478e-ba59-24cc1f2878a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:54.766893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e8a0a334-6c4c-478e-ba59-24cc1f2878a5 None None] MFA Rules not processed for user `13af20e11c9449fd807eeeb1b3ff898f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:54.768092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8a0a334-6c4c-478e-ba59-24cc1f2878a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:54.825629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1301/2580] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:01:55.107624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:55.114941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:55.167201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:55.181348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb4cd-dff4-4f10-8f32-4d19088cdd8e service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:55.185002 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7608f3c-73fd-4b5d-9be1-bc752f633eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:55.185624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7608f3c-73fd-4b5d-9be1-bc752f633eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:55.185908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7608f3c-73fd-4b5d-9be1-bc752f633eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:55.186184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7608f3c-73fd-4b5d-9be1-bc752f633eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:55.188053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e7608f3c-73fd-4b5d-9be1-bc752f633eb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:55.200232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7608f3c-73fd-4b5d-9be1-bc752f633eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:55.201734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7608f3c-73fd-4b5d-9be1-bc752f633eb3 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:55.210500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1280/2581] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:01:57.705525 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a334-6c4c-478e-ba59-24cc1f2878a5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:57.821489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a334-6c4c-478e-ba59-24cc1f2878a5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:57.826953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:57.827844 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:57.828027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:57.828144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:57.828613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] PATH_INFO: `/v3/users/b49a46ec0ca54316a76632f7adabf55f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:57.828731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:57.828731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:57.829512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49a46ec0ca54316a76632f7adabf55f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:57.832160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:57.850014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:57.851227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:57.864281 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:57.890706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49a46ec0ca54316a76632f7adabf55f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:57.899444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49a46ec0ca54316a76632f7adabf55f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:57.918128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49a46ec0ca54316a76632f7adabf55f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:57.921741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1281/2582] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 29 07:01:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:57.934085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49a46ec0ca54316a76632f7adabf55f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:57.934722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f2ce113-2550-4178-9d3e-57cbecb624b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49a46ec0ca54316a76632f7adabf55f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:57.935025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:57.936704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:57.941881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1302/2583] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:57 2022] DELETE /identity/v3/users/b49a46ec0ca54316a76632f7adabf55f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:57.946201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:57.946490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:57.946769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:57.966573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:01:57.967690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:58.029850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6ce1-6268-41c7-97fe-31257092bc69 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:58.036766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1303/2584] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:01:58.046502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:58.047940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:58.110918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d153c-907f-4f51-b2d3-83732b4a15c6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:58.112638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:58.113346 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:58.113555 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:58.113830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:58.115577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:58.115840 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:58.116335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:58.116720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:58.117072 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:58.127183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:58.129636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:58.140154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1304/2585] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:01:58.154272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:58.187679 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:58.189364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:58.199044 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1282/2586] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 07:01:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 07:01:58.665684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:58.667192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:58.688887 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fe129-e890-4457-8b19-9fb8f8263140 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:58.704699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:58.705373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:58.705665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:58.706335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] PATH_INFO: `/v3/projects/3191da1d6aa84528b76f8be8c7923c66` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:58.710722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3191da1d6aa84528b76f8be8c7923c66)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:58.712723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:58.782029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:01:58.795405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3191da1d6aa84528b76f8be8c7923c66', 'request_id': 'req-7022ec59-4ead-4868-a898-4e257542a80e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:58.806564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3191da1d6aa84528b76f8be8c7923c66', 'request_id': 'req-7022ec59-4ead-4868-a898-4e257542a80e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:58.818878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3191da1d6aa84528b76f8be8c7923c66', 'request_id': 'req-7022ec59-4ead-4868-a898-4e257542a80e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:58.837241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] The token cache is being invalidate because project 3191da1d6aa84528b76f8be8c7923c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:58.838151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:58.839081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:58.842025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1305/2587] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:58 2022] DELETE /identity/v3/projects/3191da1d6aa84528b76f8be8c7923c66 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:58.846011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:58.871102 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a030bf-b133-4d38-b182-6a2a428b36ac service swift] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:59.009851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:59.010823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:59.010823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:59.011050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] PATH_INFO: `/v3/users/26c2c97354bc43dbb44f5a0dfcf1cf84` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:59.012214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c2c97354bc43dbb44f5a0dfcf1cf84)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:59.014803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:59.117028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c2c97354bc43dbb44f5a0dfcf1cf84'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.118958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c2c97354bc43dbb44f5a0dfcf1cf84'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.154008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c2c97354bc43dbb44f5a0dfcf1cf84'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.176352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c2c97354bc43dbb44f5a0dfcf1cf84'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.179224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c2c97354bc43dbb44f5a0dfcf1cf84'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.211170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1283/2588] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:58 2022] DELETE /identity/v3/users/26c2c97354bc43dbb44f5a0dfcf1cf84 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:59.649230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:59.653435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:59.700697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022ec59-4ead-4868-a898-4e257542a80e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:01:59.803998 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:01:59.804943 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:01:59.805310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:01:59.805659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] PATH_INFO: `/v3/projects/0af1d349a6dd4116bee3821f09b445a5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:01:59.807082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af1d349a6dd4116bee3821f09b445a5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:01:59.809613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:01:59.874962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af1d349a6dd4116bee3821f09b445a5', 'request_id': 'req-5235c94a-1f65-4b75-9676-6dd20bea21be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.887604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af1d349a6dd4116bee3821f09b445a5', 'request_id': 'req-5235c94a-1f65-4b75-9676-6dd20bea21be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.897078 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af1d349a6dd4116bee3821f09b445a5', 'request_id': 'req-5235c94a-1f65-4b75-9676-6dd20bea21be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.912648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] The token cache is being invalidate because project 0af1d349a6dd4116bee3821f09b445a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:01:59.913060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.913882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:01:59.916752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1306/2589] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:01:59 2022] DELETE /identity/v3/projects/0af1d349a6dd4116bee3821f09b445a5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:01:59.920622 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:01:59.922217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:01:59.935826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12797e12-735e-41f1-ad0b-dba8f5f3882e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:00.004815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:00.028818 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:00.029498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:00.029787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:00.030071 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] PATH_INFO: `/v3/users/05e4a8203569439888686fa087edd7fd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:00.031307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e4a8203569439888686fa087edd7fd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:00.033330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:00.146770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e4a8203569439888686fa087edd7fd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.147455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e4a8203569439888686fa087edd7fd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.173172 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e4a8203569439888686fa087edd7fd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.202917 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e4a8203569439888686fa087edd7fd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.205025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-38dccdc0-3efd-41d5-a5ba-6e06c8f1f915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e4a8203569439888686fa087edd7fd'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.234004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1284/2590] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:01:59 2022] DELETE /identity/v3/users/05e4a8203569439888686fa087edd7fd => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:00.723924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:00.725748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:00.755342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235c94a-1f65-4b75-9676-6dd20bea21be admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:00.842622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:00.854829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:00.855771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:00.856427 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:00.856984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] PATH_INFO: `/v3/projects/df7d0786bbe74b5d98b70e39848f0e60` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:00.858472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7d0786bbe74b5d98b70e39848f0e60)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:00.861006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:00.915655 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7d0786bbe74b5d98b70e39848f0e60', 'request_id': 'req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.924484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7d0786bbe74b5d98b70e39848f0e60', 'request_id': 'req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.931815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7d0786bbe74b5d98b70e39848f0e60', 'request_id': 'req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.942871 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] The token cache is being invalidate because project df7d0786bbe74b5d98b70e39848f0e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:02:00.943216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.943842 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-80c88a3d-b68c-4f1b-b32e-42c93b33cf46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:00.945605 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1307/2591] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:02:00 2022] DELETE /identity/v3/projects/df7d0786bbe74b5d98b70e39848f0e60 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:00.950453 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6112c3-f4a9-4499-a9f5-fe7de65760e3 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:00.951297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6112c3-f4a9-4499-a9f5-fe7de65760e3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:00.951766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6112c3-f4a9-4499-a9f5-fe7de65760e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:00.974301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7d6112c3-f4a9-4499-a9f5-fe7de65760e3 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:00.975816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7d6112c3-f4a9-4499-a9f5-fe7de65760e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:01.042983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1285/2592] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:01.051332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504749-093d-475d-ab22-edd2cbe616c5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.051750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504749-093d-475d-ab22-edd2cbe616c5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.052116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504749-093d-475d-ab22-edd2cbe616c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.072933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aa504749-093d-475d-ab22-edd2cbe616c5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:01.074172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa504749-093d-475d-ab22-edd2cbe616c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:01.085249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1308/2593] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:01.089559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6112c3-f4a9-4499-a9f5-fe7de65760e3 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:01.102310 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6112c3-f4a9-4499-a9f5-fe7de65760e3 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:01.107018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:01.107651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.107934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.108212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.108954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:02:01.109276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:01.110876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:01.126001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1286/2594] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 07:02:01.131185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504749-093d-475d-ab22-edd2cbe616c5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:01.141668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504749-093d-475d-ab22-edd2cbe616c5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:01.147230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:01.147859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.148143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.148418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.149137 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:01.151046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:01.186178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1309/2595] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:02:01 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:01.187647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:01.198844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef44f0d-b6ab-418e-8afb-1a989aac5018 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:01.263261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:01.270747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:01.271391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.272117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.272641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.273468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:01.275562 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:01.298530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1287/2596] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:02:01 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:01.304090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:01.315596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaf308-5723-40e5-b8de-b7d58f708e92 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:01.320414 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:01.321206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.321561 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.321937 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.322707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:01.324439 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:01.337331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1310/2597] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:01.343437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:01.354784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd0ce-aebf-479b-af02-d77fc909fa3f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:01.368835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb7d0-0498-4fc5-92cf-aad602e993f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:01.368835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb7d0-0498-4fc5-92cf-aad602e993f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.368835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb7d0-0498-4fc5-92cf-aad602e993f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.368835 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb7d0-0498-4fc5-92cf-aad602e993f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.369303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb7d0-0498-4fc5-92cf-aad602e993f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:01.369303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb7d0-0498-4fc5-92cf-aad602e993f2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:01.385778 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1288/2598] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:01.389740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:01.400465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6272fb8-5379-4672-962d-bcd502ee8ada admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:01.405781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:01.406537 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.406889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.407238 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] PATH_INFO: `/v3/projects/97db3698d5114361bb0702a6a64fd806/users/3cb56da574304d69b695fa0b56d0d73d/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.423448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97db3698d5114361bb0702a6a64fd806, user_id=3cb56da574304d69b695fa0b56d0d73d, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:01.426831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:01.441041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1311/2599] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:01 2022] PUT /identity/v3/projects/97db3698d5114361bb0702a6a64fd806/users/3cb56da574304d69b695fa0b56d0d73d/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:01.452086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd306-18f3-4cd3-8576-cf93fdc750f8 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.452380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd306-18f3-4cd3-8576-cf93fdc750f8 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.452697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd306-18f3-4cd3-8576-cf93fdc750f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.493617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-aa8cd306-18f3-4cd3-8576-cf93fdc750f8 None None] MFA Rules not processed for user `3cb56da574304d69b695fa0b56d0d73d`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:01.495161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-aa8cd306-18f3-4cd3-8576-cf93fdc750f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:01.581906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1289/2600] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:01.593705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:01.596149 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:01.692646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921e01c-c6ac-4575-883a-2d79fbf01534 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:01.696464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe6ff8-3b69-4d3a-a961-533cbe69fd0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:01.697222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6ff8-3b69-4d3a-a961-533cbe69fd0d service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:01.697599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6ff8-3b69-4d3a-a961-533cbe69fd0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:01.697974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6ff8-3b69-4d3a-a961-533cbe69fd0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:01.708854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe6ff8-3b69-4d3a-a961-533cbe69fd0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:01.710442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe6ff8-3b69-4d3a-a961-533cbe69fd0d service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:01.721044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1312/2601] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:02:10.875598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd306-18f3-4cd3-8576-cf93fdc750f8 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:10.940342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd306-18f3-4cd3-8576-cf93fdc750f8 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:10.945269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:10.945923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:10.946332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:10.946623 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] PATH_INFO: `/v3/users/3cb56da574304d69b695fa0b56d0d73d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:10.947860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb56da574304d69b695fa0b56d0d73d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:10.949805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:11.002323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb56da574304d69b695fa0b56d0d73d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.002996 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb56da574304d69b695fa0b56d0d73d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.017190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb56da574304d69b695fa0b56d0d73d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.019589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:11.030787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb56da574304d69b695fa0b56d0d73d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.031894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb56da574304d69b695fa0b56d0d73d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.046617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1290/2602] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:02:10 2022] DELETE /identity/v3/users/3cb56da574304d69b695fa0b56d0d73d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:11.051912 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.052502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.052657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.071933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:11.073215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:11.094964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:11.137259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1313/2603] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:11.147231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:11.148770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:11.209353 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83892b9-0da7-4612-82b6-834a50b3fb39 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:11.213517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854b6ee-7ae0-43dd-bad3-537abcc9ce6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:11.214190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854b6ee-7ae0-43dd-bad3-537abcc9ce6e service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.214481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854b6ee-7ae0-43dd-bad3-537abcc9ce6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.214759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854b6ee-7ae0-43dd-bad3-537abcc9ce6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.225441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854b6ee-7ae0-43dd-bad3-537abcc9ce6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:11.226903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854b6ee-7ae0-43dd-bad3-537abcc9ce6e service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:11.236476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1291/2604] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:11.503831 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:11.505584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:11.521913 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce663a4-fb32-49b1-adbd-cf7e4e009248 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:11.526617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:11.527410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.527695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.527973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] PATH_INFO: `/v3/projects/97db3698d5114361bb0702a6a64fd806` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.529347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=97db3698d5114361bb0702a6a64fd806)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:11.531300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:11.577719 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97db3698d5114361bb0702a6a64fd806', 'request_id': 'req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.584748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97db3698d5114361bb0702a6a64fd806', 'request_id': 'req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.591375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97db3698d5114361bb0702a6a64fd806', 'request_id': 'req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.602456 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] The token cache is being invalidate because project 97db3698d5114361bb0702a6a64fd806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:02:11.602743 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.603381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00b3f754-7c1d-4d9d-a8e4-35a0e657b6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:11.604975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1314/2605] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:02:11 2022] DELETE /identity/v3/projects/97db3698d5114361bb0702a6a64fd806 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:11.609914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09c3b-30e2-4fa7-8a00-d6b0ef8eba86 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.610203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09c3b-30e2-4fa7-8a00-d6b0ef8eba86 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.610483 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09c3b-30e2-4fa7-8a00-d6b0ef8eba86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.631111 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-50c09c3b-30e2-4fa7-8a00-d6b0ef8eba86 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:11.632309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-50c09c3b-30e2-4fa7-8a00-d6b0ef8eba86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:11.692936 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1292/2606] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:11.701155 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba6f6e-bb81-4c3a-afce-d83395da35ee None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.701458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba6f6e-bb81-4c3a-afce-d83395da35ee None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.701745 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba6f6e-bb81-4c3a-afce-d83395da35ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.725821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bcba6f6e-bb81-4c3a-afce-d83395da35ee None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:11.727120 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bcba6f6e-bb81-4c3a-afce-d83395da35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:11.736872 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1315/2607] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:11.741446 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09c3b-30e2-4fa7-8a00-d6b0ef8eba86 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:11.756739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09c3b-30e2-4fa7-8a00-d6b0ef8eba86 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:11.766166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:11.766963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.767328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.767787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.770078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:02:11.770464 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:11.772251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:11.795875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1293/2608] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:02:11.797398 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba6f6e-bb81-4c3a-afce-d83395da35ee None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:11.807444 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba6f6e-bb81-4c3a-afce-d83395da35ee None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:11.812095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:11.812736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.813607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.813607 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.813978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:11.816436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:11.846730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1316/2609] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:02:11 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:11.851075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:11.860965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffccc0-1f14-44a9-a9f9-4c594d43fad3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:11.915919 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:11.916635 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.916824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.917098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.917756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:11.919682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:11.941252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1294/2610] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:02:11 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:11.946034 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:11.956991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580585b9-a1b4-4d10-a728-11f0ad383ec8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:11.961600 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:11.962205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.962487 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.963021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.963701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:11.965321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:11.975649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1317/2611] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:11.979607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:11.985930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:11.990455 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a03f08-a9e0-4d87-8de8-310f2b22847c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:11.996664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c223eb-80e5-4bf6-ace3-c986031aee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:11.997444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c223eb-80e5-4bf6-ace3-c986031aee92 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:11.997921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c223eb-80e5-4bf6-ace3-c986031aee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:11.998380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c223eb-80e5-4bf6-ace3-c986031aee92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:11.999068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c223eb-80e5-4bf6-ace3-c986031aee92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:12.000733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c223eb-80e5-4bf6-ace3-c986031aee92 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:12.011855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1295/2612] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:12.015910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:12.025987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23c7bb-0c74-4894-a595-20247ae8883c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:12.030670 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:12.031312 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:12.031592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:12.031867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] PATH_INFO: `/v3/projects/17de3e5c46bb43a3973d8b9728c797b7/users/f0f388a7fdbc4925940a5e1a53f15f9e/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:12.046166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17de3e5c46bb43a3973d8b9728c797b7, user_id=f0f388a7fdbc4925940a5e1a53f15f9e, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:12.048815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:12.062345 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1318/2613] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:12 2022] PUT /identity/v3/projects/17de3e5c46bb43a3973d8b9728c797b7/users/f0f388a7fdbc4925940a5e1a53f15f9e/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:12.068289 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac91f4-1432-4e7c-8d2b-10bdf8d5752c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:12.068597 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac91f4-1432-4e7c-8d2b-10bdf8d5752c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:12.068876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac91f4-1432-4e7c-8d2b-10bdf8d5752c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:12.087142 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-acac91f4-1432-4e7c-8d2b-10bdf8d5752c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:12.088626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-acac91f4-1432-4e7c-8d2b-10bdf8d5752c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:12.145378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1296/2614] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:12.155196 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:12.157272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:12.211590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:12.218262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a824ad4-31af-4aa3-b0d7-ed55395e9545 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:12.223786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:12.224410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:12.224739 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:12.225082 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:12.235276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:12.237517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:12.247860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1319/2615] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:16.639365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732d0d-e7da-42be-81dd-5e3444f057ef None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:16.639847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732d0d-e7da-42be-81dd-5e3444f057ef None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:16.639939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732d0d-e7da-42be-81dd-5e3444f057ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:16.682192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-3d732d0d-e7da-42be-81dd-5e3444f057ef None None] MFA Rules not processed for user `f0f388a7fdbc4925940a5e1a53f15f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:16.683500 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3d732d0d-e7da-42be-81dd-5e3444f057ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:16.781658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1297/2616] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:16.783673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:16.794678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea9a69-f93b-4952-ae42-75b03899b1f5 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:16.800352 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:16.801016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:16.801296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:16.801570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:16.802448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:16.804356 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:16.848581 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d732d0d-e7da-42be-81dd-5e3444f057ef None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:16.850235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1320/2617] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:02:16 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:16.872724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d732d0d-e7da-42be-81dd-5e3444f057ef None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:16.937658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:16.938340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:16.938719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:16.939089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:16.940417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:16.942456 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:16.982846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1298/2618] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:02:16 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:02:16.984048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:17.003863 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af4518-2bf2-4172-9554-2586704c747f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:17.008815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:17.009571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:17.009976 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:17.010381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:17.011167 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:17.013054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:17.028633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1321/2619] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:17.030204 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:17.042409 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e63100-aa49-4fc4-9a13-75ca14b553ed admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:17.048595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:17.049596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:17.049879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:17.050094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] PATH_INFO: `/v3/projects/f14ec3dd113b4c3e842abdf9a3c4fcf1/users/18450b80ca4b44a5b3db1babbdc0baef/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:17.066190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:17.071389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14ec3dd113b4c3e842abdf9a3c4fcf1, user_id=18450b80ca4b44a5b3db1babbdc0baef, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:17.074480 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:17.090604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1299/2620] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:17 2022] PUT /identity/v3/projects/f14ec3dd113b4c3e842abdf9a3c4fcf1/users/18450b80ca4b44a5b3db1babbdc0baef/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:17.094608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:17.105756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d25a65-526f-4ec3-90f8-ff505ee92f8e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:17.163970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaac93-7870-4a1c-bfeb-ce29b76d8dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:17.164634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaac93-7870-4a1c-bfeb-ce29b76d8dff admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:17.164952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaac93-7870-4a1c-bfeb-ce29b76d8dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:17.165234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaac93-7870-4a1c-bfeb-ce29b76d8dff admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:17.165924 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaac93-7870-4a1c-bfeb-ce29b76d8dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:17.167508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaac93-7870-4a1c-bfeb-ce29b76d8dff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:17.178819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1322/2621] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:17.182921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:17.194259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d9225-b3c4-4bdc-9a72-bb6c83c5baa1 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:17.199479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:17.200149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:17.200428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:17.200989 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] PATH_INFO: `/v3/projects/f14ec3dd113b4c3e842abdf9a3c4fcf1/users/18450b80ca4b44a5b3db1babbdc0baef/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:17.202860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14ec3dd113b4c3e842abdf9a3c4fcf1, user_id=18450b80ca4b44a5b3db1babbdc0baef, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:17.205946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:17.216875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1300/2622] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:17 2022] PUT /identity/v3/projects/f14ec3dd113b4c3e842abdf9a3c4fcf1/users/18450b80ca4b44a5b3db1babbdc0baef/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:22.159193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a579-1778-4ec5-a171-20f9ff6be695 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:22.159710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a579-1778-4ec5-a171-20f9ff6be695 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:22.159879 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a579-1778-4ec5-a171-20f9ff6be695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:22.190146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-38d7a579-1778-4ec5-a171-20f9ff6be695 None None] MFA Rules not processed for user `18450b80ca4b44a5b3db1babbdc0baef`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:22.191452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-38d7a579-1778-4ec5-a171-20f9ff6be695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:22.268222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1323/2623] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:22.282163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:22.285573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:22.362410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1d16-1a37-4914-8bb9-2b1bae8ec054 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:22.368612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:22.369256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:22.369560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:22.369834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:22.430728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:22.432814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:22.456280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1301/2624] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:25.925900 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7a579-1778-4ec5-a171-20f9ff6be695 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:25.973373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:25.990585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7a579-1778-4ec5-a171-20f9ff6be695 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:25.994624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:25.995399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:25.995801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:25.996179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:25.999418 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:26.076686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:26.078159 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:26.091094 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1324/2625] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:28.046157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:28.062410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb82ca1-1ea0-48ac-aacf-a08aba375f6e service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:28.073593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:28.074238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:28.074540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:28.074792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:28.076615 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:28.092690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:28.094445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:28.107024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1302/2626] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:29.246246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:29.247802 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:29.260789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba6bc1-b5d4-4d27-9f00-4b2792191845 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:29.264497 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b53a59-1f8d-4f8c-a527-c6a85e589a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:29.265246 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b53a59-1f8d-4f8c-a527-c6a85e589a5a service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:29.265538 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b53a59-1f8d-4f8c-a527-c6a85e589a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:29.265817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b53a59-1f8d-4f8c-a527-c6a85e589a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:29.267585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f7b53a59-1f8d-4f8c-a527-c6a85e589a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:29.281391 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b53a59-1f8d-4f8c-a527-c6a85e589a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:29.282874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b53a59-1f8d-4f8c-a527-c6a85e589a5a service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:29.294795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1325/2627] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:29.870258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:29.880120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41826f6-f1bf-454b-a246-cb76b68fc14f service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:29.884332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:29.885326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:29.885825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:29.886314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:29.888424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:29.919674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:29.921959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:29.937908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1303/2628] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:02:38.797343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:38.797888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:38.797888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:38.818143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] MFA Rules not processed for user `f0f388a7fdbc4925940a5e1a53f15f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:38.819337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:38.826301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1326/2629] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:39.932632 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=77,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:48.812189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=42 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:50.066239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:50.075416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef7da4-a3ee-4f81-9e6c-ba421b226c67 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:50.079290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:50.079911 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:50.080196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:50.080504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:50.082340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:50.147617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:50.149108 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:50.160230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1304/2630] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:53.261818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:53.263369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:53.285932 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2135b0c-eb79-491b-8de2-0be7c499bf71 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:53.294811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:53.295859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:53.296571 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:53.296981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] PATH_INFO: `/v3/users/12a751b41fd546ac9017c67d6de4ebdd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:53.298286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a751b41fd546ac9017c67d6de4ebdd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:53.300276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:53.363378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a751b41fd546ac9017c67d6de4ebdd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.371733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a751b41fd546ac9017c67d6de4ebdd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.393973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a751b41fd546ac9017c67d6de4ebdd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.407468 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a751b41fd546ac9017c67d6de4ebdd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.408365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-54780652-775d-48af-8d7e-00ce7cb7609c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a751b41fd546ac9017c67d6de4ebdd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.431526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1327/2631] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:02:53 2022] DELETE /identity/v3/users/12a751b41fd546ac9017c67d6de4ebdd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:53.640858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:53.653042 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c3bf-8849-44d1-a652-51c612e7fd48 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:53.751037 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:53.751694 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:53.751981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:53.752259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] PATH_INFO: `/v3/projects/10b54d7ff9644981893f886e59f22021` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:53.753595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b54d7ff9644981893f886e59f22021)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:53.755498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:53.854712 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b54d7ff9644981893f886e59f22021', 'request_id': 'req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.861846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b54d7ff9644981893f886e59f22021', 'request_id': 'req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.876099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b54d7ff9644981893f886e59f22021', 'request_id': 'req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.890805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] The token cache is being invalidate because project 10b54d7ff9644981893f886e59f22021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:02:53.891104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.891766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-6bde0bbb-770a-4fff-b9ab-a7aebadfccc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:53.893337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1305/2632] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:02:53 2022] DELETE /identity/v3/projects/10b54d7ff9644981893f886e59f22021 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:53.898636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2a2e-881b-4492-8a70-8eb6639d6fe2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:53.899048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2a2e-881b-4492-8a70-8eb6639d6fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:53.899404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2a2e-881b-4492-8a70-8eb6639d6fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:53.927170 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aced2a2e-881b-4492-8a70-8eb6639d6fe2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:53.929219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aced2a2e-881b-4492-8a70-8eb6639d6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:53.991407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1328/2633] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:53.999863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cbd70-65cd-45f7-9fb6-ca16b03ed7e7 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.000149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cbd70-65cd-45f7-9fb6-ca16b03ed7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.000425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cbd70-65cd-45f7-9fb6-ca16b03ed7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.025513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-021cbd70-65cd-45f7-9fb6-ca16b03ed7e7 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:54.026719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-021cbd70-65cd-45f7-9fb6-ca16b03ed7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:54.034564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1306/2634] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:54.039906 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced2a2e-881b-4492-8a70-8eb6639d6fe2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:54.049940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced2a2e-881b-4492-8a70-8eb6639d6fe2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:54.055310 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:54.055919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.056206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.056485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.057205 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:02:54.057512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:54.059099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:54.071123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1329/2635] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:02:54.075468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cbd70-65cd-45f7-9fb6-ca16b03ed7e7 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:54.085143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cbd70-65cd-45f7-9fb6-ca16b03ed7e7 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:54.090065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:54.090663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.090942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.091215 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.091855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:54.093765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:54.108946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:54.122970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1307/2636] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:02:54 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:54.127117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:54.137441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac13056-91be-4ba1-83b6-e821ec2cbbbf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:54.199190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:54.199967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.200362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.200799 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.201578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:54.203578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:54.211955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:54.232458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1330/2637] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:02:54 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:54.236293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:54.248646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906ab04-8813-4d66-a393-244be4e80605 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:54.253816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:54.254512 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.254889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.255236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.256673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:54.258423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:54.268828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1308/2638] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:54.273989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:54.290920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d4f80-8c57-4a50-857a-fe7e2dd4a3ff admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:54.296813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6ebdb-109b-4bda-9cb9-52e4352c63c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:54.297564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ebdb-109b-4bda-9cb9-52e4352c63c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.297852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ebdb-109b-4bda-9cb9-52e4352c63c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.298127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6ebdb-109b-4bda-9cb9-52e4352c63c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.298811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6ebdb-109b-4bda-9cb9-52e4352c63c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:54.300411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6ebdb-109b-4bda-9cb9-52e4352c63c1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:54.313090 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1331/2639] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:54.317107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:54.329624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99510c-cd9f-483b-b519-dfa0c50cb24a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:54.334909 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:54.335633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.335925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.336218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] PATH_INFO: `/v3/projects/c7872e10f1524a4790ddcbe1956a50c9/users/8ab39e1095374504bfb88e6da98d66eb/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.363501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7872e10f1524a4790ddcbe1956a50c9, user_id=8ab39e1095374504bfb88e6da98d66eb, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:54.366938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:54.381836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1309/2640] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:54 2022] PUT /identity/v3/projects/c7872e10f1524a4790ddcbe1956a50c9/users/8ab39e1095374504bfb88e6da98d66eb/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:54.386856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2ec4-caaf-49d0-8397-a450c7378f97 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.387195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2ec4-caaf-49d0-8397-a450c7378f97 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.387470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2ec4-caaf-49d0-8397-a450c7378f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.405720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-378b2ec4-caaf-49d0-8397-a450c7378f97 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:54.406902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-378b2ec4-caaf-49d0-8397-a450c7378f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:54.476724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1332/2641] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:54.481196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:54.482955 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:54.547972 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077eb3ab-b906-44e7-a9c7-13996b322a49 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:54.551646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:54.552272 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:54.552577 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:54.553501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:54.562981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:54.564410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:54.580723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1310/2642] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:02:55.255102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27700ca-10c5-4b98-90f4-a72e258e4db0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.255471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27700ca-10c5-4b98-90f4-a72e258e4db0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.255850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27700ca-10c5-4b98-90f4-a72e258e4db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.291177 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d27700ca-10c5-4b98-90f4-a72e258e4db0 None None] MFA Rules not processed for user `8ab39e1095374504bfb88e6da98d66eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:55.292656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d27700ca-10c5-4b98-90f4-a72e258e4db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:55.317613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:55.319926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:55.336639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139007ad-e8b8-400f-8c05-35ec41c49d3b service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:55.341099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:55.341812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.342116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.342395 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.344214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:55.389677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1333/2643] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:55.390792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27700ca-10c5-4b98-90f4-a72e258e4db0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:55.409287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27700ca-10c5-4b98-90f4-a72e258e4db0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:55.415671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:55.417297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.418096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.418604 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.418727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:55.419637 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:55.420384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:55.422011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:55.432334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1311/2644] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:02:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:02:55.457121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1334/2645] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:02:55 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:55.461149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:55.471926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef99a-c518-48cd-b8ac-0d9f563e94bc service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:55.547374 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:55.555382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:55.556138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.556652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.557046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.557859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:55.560503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:55.585830 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1312/2646] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:02:55 2022] POST /identity/v3/users => generated 308 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:55.590197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:55.601234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73144a-b91a-458f-987a-97dc009203c4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:55.606194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:55.606850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.607132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.607582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.608283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:55.609927 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:55.621774 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1335/2647] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:55.626126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:55.638000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b37e-1c52-4276-8a6f-f4556ae7146d admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:55.642980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:55.643673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.643956 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.644238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] PATH_INFO: `/v3/projects/6853c29d071647baa6958cc3c5a301b8/users/e4d41b5c15d14d689e430b37fb6c174c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.662550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6853c29d071647baa6958cc3c5a301b8, user_id=e4d41b5c15d14d689e430b37fb6c174c, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:55.665676 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:55.680583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1313/2648] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:55 2022] PUT /identity/v3/projects/6853c29d071647baa6958cc3c5a301b8/users/e4d41b5c15d14d689e430b37fb6c174c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:55.684813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:55.696105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636fdec-3e46-49e7-9bcb-803187beafa4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:55.732055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:55.761169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee4206-d07a-4661-a7dc-c4331dcff2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:55.761834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4206-d07a-4661-a7dc-c4331dcff2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.762119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4206-d07a-4661-a7dc-c4331dcff2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.762392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4206-d07a-4661-a7dc-c4331dcff2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.763086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee4206-d07a-4661-a7dc-c4331dcff2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:55.764737 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee4206-d07a-4661-a7dc-c4331dcff2cc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:55.775808 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1336/2649] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:55.780152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:55.791451 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ab3b-6a22-4dec-88ae-a9f17573769e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:55.796317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:55.797112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:55.797487 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:55.797833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] PATH_INFO: `/v3/projects/6853c29d071647baa6958cc3c5a301b8/users/e4d41b5c15d14d689e430b37fb6c174c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:55.799858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6853c29d071647baa6958cc3c5a301b8, user_id=e4d41b5c15d14d689e430b37fb6c174c, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:55.802646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:55.814727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1314/2650] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:55 2022] PUT /identity/v3/projects/6853c29d071647baa6958cc3c5a301b8/users/e4d41b5c15d14d689e430b37fb6c174c/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:56.916829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ab77-ce33-4526-9b37-73275d31945b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:56.916829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ab77-ce33-4526-9b37-73275d31945b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:56.916829 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ab77-ce33-4526-9b37-73275d31945b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:56.950030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1a77ab77-ce33-4526-9b37-73275d31945b None None] MFA Rules not processed for user `e4d41b5c15d14d689e430b37fb6c174c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:56.951273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a77ab77-ce33-4526-9b37-73275d31945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:57.043133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1337/2651] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:56 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:58.180574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:58.261991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7888e8-574d-4347-8262-222c44bb8142 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:58.268191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:58.269085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:58.269983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:58.270439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] PATH_INFO: `/v3/users/8ab39e1095374504bfb88e6da98d66eb` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:58.272572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab39e1095374504bfb88e6da98d66eb)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:58.274498 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:58.328341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab39e1095374504bfb88e6da98d66eb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.329127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab39e1095374504bfb88e6da98d66eb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.343088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab39e1095374504bfb88e6da98d66eb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.350507 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab39e1095374504bfb88e6da98d66eb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.351529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab39e1095374504bfb88e6da98d66eb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.370160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1315/2652] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:02:58 2022] DELETE /identity/v3/users/8ab39e1095374504bfb88e6da98d66eb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:58.565835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77ab77-ce33-4526-9b37-73275d31945b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:58.589660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77ab77-ce33-4526-9b37-73275d31945b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:58.652955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:58.653713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:58.654185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:58.654467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] PATH_INFO: `/v3/projects/c7872e10f1524a4790ddcbe1956a50c9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:58.655695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7872e10f1524a4790ddcbe1956a50c9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:58.657692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:58.708690 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7872e10f1524a4790ddcbe1956a50c9', 'request_id': 'req-662ed930-c523-465e-9292-303cfb797e09'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.717443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7872e10f1524a4790ddcbe1956a50c9', 'request_id': 'req-662ed930-c523-465e-9292-303cfb797e09'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.720720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:58.725942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7872e10f1524a4790ddcbe1956a50c9', 'request_id': 'req-662ed930-c523-465e-9292-303cfb797e09'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.739237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] The token cache is being invalidate because project c7872e10f1524a4790ddcbe1956a50c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:02:58.739533 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.740883 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.742492 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1338/2653] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:02:58 2022] DELETE /identity/v3/projects/c7872e10f1524a4790ddcbe1956a50c9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:58.746665 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:58.748104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:58.761312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334c87-6772-4d5a-8d7f-417049cdef11 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:58.788140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:58.821490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:58.822259 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:58.822574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:58.822855 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] PATH_INFO: `/v3/users/e4d41b5c15d14d689e430b37fb6c174c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:58.824074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d41b5c15d14d689e430b37fb6c174c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:58.826166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:58.878748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d41b5c15d14d689e430b37fb6c174c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.879650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d41b5c15d14d689e430b37fb6c174c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.892112 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d41b5c15d14d689e430b37fb6c174c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.899998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d41b5c15d14d689e430b37fb6c174c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.900795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a0220508-b71d-4367-a7ab-e0819e712864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d41b5c15d14d689e430b37fb6c174c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:58.915484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1316/2654] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:02:58 2022] DELETE /identity/v3/users/e4d41b5c15d14d689e430b37fb6c174c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:59.093051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.094584 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:59.107464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ed930-c523-465e-9292-303cfb797e09 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.169416 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.170068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.170354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.170636 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] PATH_INFO: `/v3/projects/6853c29d071647baa6958cc3c5a301b8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.171824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6853c29d071647baa6958cc3c5a301b8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.173814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.219983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6853c29d071647baa6958cc3c5a301b8', 'request_id': 'req-697a74b4-602e-44d7-9e39-9cf2db5db93d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:59.226941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6853c29d071647baa6958cc3c5a301b8', 'request_id': 'req-697a74b4-602e-44d7-9e39-9cf2db5db93d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:59.233193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6853c29d071647baa6958cc3c5a301b8', 'request_id': 'req-697a74b4-602e-44d7-9e39-9cf2db5db93d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:59.243738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] The token cache is being invalidate because project 6853c29d071647baa6958cc3c5a301b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:02:59.244027 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:59.244705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-697a74b4-602e-44d7-9e39-9cf2db5db93d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:02:59.246543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1339/2655] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:02:59 2022] DELETE /identity/v3/projects/6853c29d071647baa6958cc3c5a301b8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:59.251088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a5051-fd77-4191-843a-6b0d6c80d221 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.251534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a5051-fd77-4191-843a-6b0d6c80d221 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.251755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a5051-fd77-4191-843a-6b0d6c80d221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.271309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-778a5051-fd77-4191-843a-6b0d6c80d221 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:59.273257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-778a5051-fd77-4191-843a-6b0d6c80d221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:59.333091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1317/2656] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:59.341779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f73b4-3038-4163-afdb-d1f36ef12f0a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.342068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f73b4-3038-4163-afdb-d1f36ef12f0a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.342346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f73b4-3038-4163-afdb-d1f36ef12f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.362207 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c70f73b4-3038-4163-afdb-d1f36ef12f0a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:59.363406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c70f73b4-3038-4163-afdb-d1f36ef12f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:59.371591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1340/2657] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:59.375807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a5051-fd77-4191-843a-6b0d6c80d221 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.387075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a5051-fd77-4191-843a-6b0d6c80d221 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.392065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.392999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.393297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.393572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.394265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:02:59.394569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.396218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.410169 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1318/2658] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:02:59.414348 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f73b4-3038-4163-afdb-d1f36ef12f0a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.423738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f73b4-3038-4163-afdb-d1f36ef12f0a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.428995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.429626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.429909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.430187 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.430843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.432840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.460074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1341/2659] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:02:59 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:59.464379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.475021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfbb34-c69c-4c39-9b43-e957abe3edf5 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.506738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:59.533685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.534327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.534606 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.534905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.535564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.538146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.558515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1319/2660] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:02:59 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:59.562575 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.576019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3456a-19ce-4d0a-9268-fa1925f7e795 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.581115 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.581730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.582011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.582284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.582975 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.585271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.595723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1342/2661] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:59.600251 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.611279 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e348a-fc99-4277-a624-81d029e3613e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.616031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25463b23-2315-4ebd-abab-3fc14a776102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.616668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25463b23-2315-4ebd-abab-3fc14a776102 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.616952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25463b23-2315-4ebd-abab-3fc14a776102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.617218 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25463b23-2315-4ebd-abab-3fc14a776102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.617898 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25463b23-2315-4ebd-abab-3fc14a776102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.619519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25463b23-2315-4ebd-abab-3fc14a776102 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.630508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1320/2662] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:02:59.634754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.644915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550422a5-6f6b-4a3a-8115-aaab0d5909ed admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.650009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.650675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.650957 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.651233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] PATH_INFO: `/v3/projects/d3f07b466a194870b8127f77f4a76f3f/users/941c355579b945699b91e6837639c9cd/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.667641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f07b466a194870b8127f77f4a76f3f, user_id=941c355579b945699b91e6837639c9cd, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.670353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.683819 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1343/2663] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:02:59 2022] PUT /identity/v3/projects/d3f07b466a194870b8127f77f4a76f3f/users/941c355579b945699b91e6837639c9cd/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:02:59.694790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8af255-40b8-4bd5-89bc-c9ca6718e5e3 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.695074 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8af255-40b8-4bd5-89bc-c9ca6718e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.695351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8af255-40b8-4bd5-89bc-c9ca6718e5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.722868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7a8af255-40b8-4bd5-89bc-c9ca6718e5e3 None None] MFA Rules not processed for user `941c355579b945699b91e6837639c9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:02:59.724057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7a8af255-40b8-4bd5-89bc-c9ca6718e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:02:59.788805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1321/2664] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:02:59.793237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.806886 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:02:59.856904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29862e-0d26-4ac9-ba6f-d65e66a2d122 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.861499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.862111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.862392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.862671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.863322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.865275 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.892938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1344/2665] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:02:59 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:02:59.897520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8af255-40b8-4bd5-89bc-c9ca6718e5e3 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:02:59.908261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8af255-40b8-4bd5-89bc-c9ca6718e5e3 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:02:59.965010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:02:59.965639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:02:59.965954 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:02:59.966230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:02:59.966902 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:02:59.968851 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:02:59.989168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1322/2666] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:02:59 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:02:59.993292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:00.003654 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b10ee-15b7-40fb-9341-5c916765c4e0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:00.009144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:00.009758 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:00.010076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:00.010382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:00.011070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:00.012720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:00.023703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1345/2667] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:00.027672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:00.038008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689815ba-e0c9-4b8b-9292-28b7ecc6df6f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:00.042625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:00.043419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:00.043749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:00.045380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] PATH_INFO: `/v3/projects/ba6d5bbe8adb48fe88de630486a878d6/users/b996c0fffe874db39aed18e4d3f30a98/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:00.063340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6d5bbe8adb48fe88de630486a878d6, user_id=b996c0fffe874db39aed18e4d3f30a98, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:00.066064 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:00.078764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1323/2668] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:00 2022] PUT /identity/v3/projects/ba6d5bbe8adb48fe88de630486a878d6/users/b996c0fffe874db39aed18e4d3f30a98/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:00.082742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:00.092985 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0bbda-fc23-4781-ba13-4b7e7891dc76 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:00.149778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a2353-8da0-4c4a-a45d-d0250f508a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:00.150322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a2353-8da0-4c4a-a45d-d0250f508a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:00.150611 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a2353-8da0-4c4a-a45d-d0250f508a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:00.150894 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a2353-8da0-4c4a-a45d-d0250f508a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:00.151586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a2353-8da0-4c4a-a45d-d0250f508a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:00.153218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a2353-8da0-4c4a-a45d-d0250f508a56 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:00.162946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1346/2669] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:00.167092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:00.178301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3671e8-998b-4144-a992-af8b98c38fdd admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:00.183097 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:00.183767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:00.184054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:00.184343 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] PATH_INFO: `/v3/projects/ba6d5bbe8adb48fe88de630486a878d6/users/b996c0fffe874db39aed18e4d3f30a98/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:00.186349 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6d5bbe8adb48fe88de630486a878d6, user_id=b996c0fffe874db39aed18e4d3f30a98, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:00.189029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:00.192558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:00.202328 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1324/2670] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:00 2022] PUT /identity/v3/projects/ba6d5bbe8adb48fe88de630486a878d6/users/b996c0fffe874db39aed18e4d3f30a98/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:00.214044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031792-ef16-4f8c-8a05-8f183a9e2f5a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:00.214335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031792-ef16-4f8c-8a05-8f183a9e2f5a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:00.214615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031792-ef16-4f8c-8a05-8f183a9e2f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:00.242725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f5031792-ef16-4f8c-8a05-8f183a9e2f5a None None] MFA Rules not processed for user `b996c0fffe874db39aed18e4d3f30a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:00.243947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f5031792-ef16-4f8c-8a05-8f183a9e2f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:00.311060 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1347/2671] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:00.323428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:00.325146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:00.382146 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e60a9-ba29-4562-a260-809f128e2a09 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:00.385831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:00.386441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:00.386726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:00.387010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:00.395938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:00.397422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:00.407025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1325/2672] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:00.486472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5031792-ef16-4f8c-8a05-8f183a9e2f5a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:00.497043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5031792-ef16-4f8c-8a05-8f183a9e2f5a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:00.500901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:00.501555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:00.501843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:00.502126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:00.508693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:00.571564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:00.573068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:00.582677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1348/2673] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:10.402657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=28 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:10.578174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=24 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:35.975723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:35.977465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:35.989631 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0fecf-de9a-477e-a29e-565aecc17cf6 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:35.993557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d5aaf-0c84-433f-896c-32aa9edb086f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:35.994863 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5aaf-0c84-433f-896c-32aa9edb086f service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:35.995157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5aaf-0c84-433f-896c-32aa9edb086f service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:35.995439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5aaf-0c84-433f-896c-32aa9edb086f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:35.997341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8e4d5aaf-0c84-433f-896c-32aa9edb086f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:36.058690 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d5aaf-0c84-433f-896c-32aa9edb086f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:36.060261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d5aaf-0c84-433f-896c-32aa9edb086f service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:36.069683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1326/2674] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:38.875271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:38.936959 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3aab9-d668-4807-b0a6-c3ee015af3e7 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:38.941458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:38.942087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:38.942366 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:38.942641 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] PATH_INFO: `/v3/users/941c355579b945699b91e6837639c9cd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:38.943779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=941c355579b945699b91e6837639c9cd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:38.945720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:38.991977 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941c355579b945699b91e6837639c9cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:38.999778 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941c355579b945699b91e6837639c9cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.011717 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941c355579b945699b91e6837639c9cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.018892 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941c355579b945699b91e6837639c9cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.019622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941c355579b945699b91e6837639c9cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.026157 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1349/2675] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:03:38 2022] DELETE /identity/v3/users/941c355579b945699b91e6837639c9cd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:39.030650 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:39.031207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:39.031207 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:39.058844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:39.060046 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:39.118313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1327/2676] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:39.128014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:39.129415 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:39.196966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae85aa-e5a0-47ea-9597-3b5e59e503d9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:39.200615 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:39.201227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:39.201506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:39.201780 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:39.210701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:39.212118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:39.222260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1350/2677] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:39.497743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:39.499292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:39.512028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f757b-f6d8-4233-a4ac-1bdcc0bf1033 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:39.517090 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:39.517742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:39.518221 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:39.518364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] PATH_INFO: `/v3/projects/d3f07b466a194870b8127f77f4a76f3f` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:39.527249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f07b466a194870b8127f77f4a76f3f)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:39.529311 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:39.577607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f07b466a194870b8127f77f4a76f3f', 'request_id': 'req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.583699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f07b466a194870b8127f77f4a76f3f', 'request_id': 'req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.591106 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f07b466a194870b8127f77f4a76f3f', 'request_id': 'req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.601537 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] The token cache is being invalidate because project d3f07b466a194870b8127f77f4a76f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:03:39.601826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.602441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.604756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1328/2678] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:03:39 2022] DELETE /identity/v3/projects/d3f07b466a194870b8127f77f4a76f3f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:03:39.608855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:39.619130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6c139-212d-453f-abc8-9c4cf1dad34c service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:39.628363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:39.680188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:39.680866 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:39.681861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:39.682182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] PATH_INFO: `/v3/users/b996c0fffe874db39aed18e4d3f30a98` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:39.683367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b996c0fffe874db39aed18e4d3f30a98)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:39.685302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:39.732318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b996c0fffe874db39aed18e4d3f30a98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.739267 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b996c0fffe874db39aed18e4d3f30a98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.757379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b996c0fffe874db39aed18e4d3f30a98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.767464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b996c0fffe874db39aed18e4d3f30a98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.768101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d22934be-46da-4e81-856c-528a20adbcce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b996c0fffe874db39aed18e4d3f30a98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:39.774142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1351/2679] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:03:39 2022] DELETE /identity/v3/users/b996c0fffe874db39aed18e4d3f30a98 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:40.076192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.077719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:40.091402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f6f7e-b030-4a70-81e6-d666c8ebe40e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.103897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:40.151702 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.152344 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.152662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.152945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] PATH_INFO: `/v3/projects/ba6d5bbe8adb48fe88de630486a878d6` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.154088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6d5bbe8adb48fe88de630486a878d6)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.156095 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.201254 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6d5bbe8adb48fe88de630486a878d6', 'request_id': 'req-0277a182-082d-4da1-b170-483a561c04d4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:40.212246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6d5bbe8adb48fe88de630486a878d6', 'request_id': 'req-0277a182-082d-4da1-b170-483a561c04d4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:40.220404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6d5bbe8adb48fe88de630486a878d6', 'request_id': 'req-0277a182-082d-4da1-b170-483a561c04d4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:40.231223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] The token cache is being invalidate because project ba6d5bbe8adb48fe88de630486a878d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:03:40.231515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:40.232170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0277a182-082d-4da1-b170-483a561c04d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:40.233738 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1329/2680] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:03:40 2022] DELETE /identity/v3/projects/ba6d5bbe8adb48fe88de630486a878d6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:40.238628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bee92-4bad-4689-8043-b1898acca748 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.239122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bee92-4bad-4689-8043-b1898acca748 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.239240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bee92-4bad-4689-8043-b1898acca748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.288092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-400bee92-4bad-4689-8043-b1898acca748 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:40.289316 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-400bee92-4bad-4689-8043-b1898acca748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:40.351623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1352/2681] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:03:40.360028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda42b3-01a4-41cb-b93a-ded6bb65fe12 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.360316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda42b3-01a4-41cb-b93a-ded6bb65fe12 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.360634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda42b3-01a4-41cb-b93a-ded6bb65fe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.380696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-8fda42b3-01a4-41cb-b93a-ded6bb65fe12 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:40.381925 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8fda42b3-01a4-41cb-b93a-ded6bb65fe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:40.390545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1330/2682] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:40.394786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bee92-4bad-4689-8043-b1898acca748 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.405748 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bee92-4bad-4689-8043-b1898acca748 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.410336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.410940 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.411225 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.411500 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.412181 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:03:40.412488 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.414103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.415583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:40.428164 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1353/2683] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:03:40.432684 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda42b3-01a4-41cb-b93a-ded6bb65fe12 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.441860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda42b3-01a4-41cb-b93a-ded6bb65fe12 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.446499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.448088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.448379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.448692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.449357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.451242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.488865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1331/2684] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:03:40 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:40.492779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.503347 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716dc9b-e5c6-4d42-848e-65da3528bcf2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.564678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.565231 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.565520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.565800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.566474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.568381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.590628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1354/2685] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:03:40 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:40.594673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.605865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1eb42e-ade7-4609-bf8f-f232409048c6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.610621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.611230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.611509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.611783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.612466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.614076 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.624180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1332/2686] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:40.627865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.637918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047adf17-48a2-4f9a-b1b6-8fbd6946b846 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.642467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193948b-b265-47b4-a855-ae984de2f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.643070 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193948b-b265-47b4-a855-ae984de2f5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.643382 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193948b-b265-47b4-a855-ae984de2f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.643663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193948b-b265-47b4-a855-ae984de2f5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.644341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193948b-b265-47b4-a855-ae984de2f5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.645962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193948b-b265-47b4-a855-ae984de2f5a4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.656513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1355/2687] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:40.660633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.671734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf38ae-b047-4afd-8b8e-687e1a4f5b72 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.676419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.677148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.677435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.677745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] PATH_INFO: `/v3/projects/4d32f6cf977e4836a6e8a8a2c9a36530/users/6866586e4aef44e78d2702a337cb97de/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.692977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d32f6cf977e4836a6e8a8a2c9a36530, user_id=6866586e4aef44e78d2702a337cb97de, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.695611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.708733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1333/2688] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:40 2022] PUT /identity/v3/projects/4d32f6cf977e4836a6e8a8a2c9a36530/users/6866586e4aef44e78d2702a337cb97de/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:40.775667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e545bc-cd82-45d6-abed-0934c2d172f5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.775945 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e545bc-cd82-45d6-abed-0934c2d172f5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.776233 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e545bc-cd82-45d6-abed-0934c2d172f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.803250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-08e545bc-cd82-45d6-abed-0934c2d172f5 None None] MFA Rules not processed for user `6866586e4aef44e78d2702a337cb97de`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:40.804455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-08e545bc-cd82-45d6-abed-0934c2d172f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:40.867057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1356/2689] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:40.872185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.907238 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:40.936443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabe17d-bc89-47d5-8d8e-ffdf15be06b1 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:40.941191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:40.941810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:40.942092 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:40.942368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:40.943036 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:40.944959 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:40.973928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1334/2690] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:03:40 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:40.978532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e545bc-cd82-45d6-abed-0934c2d172f5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:40.989679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e545bc-cd82-45d6-abed-0934c2d172f5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:41.047755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:41.048372 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.048692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.048974 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.049632 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:41.051548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:41.060595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:41.073684 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1357/2691] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:03:40 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:41.077707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:41.089775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bccc5-10ae-4be7-8d14-d04c9c526ca3 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:41.094580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:41.095192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.095479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.095759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.096453 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:41.098065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:41.108814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1335/2692] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:41.112899 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:41.123151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e3164-b321-48fd-8867-747900f15e07 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:41.127687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:41.128344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.128662 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.128941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] PATH_INFO: `/v3/projects/b788486b75734c37946606125578aefa/users/001f4f2434334699ad83b86e54d9664e/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.143962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b788486b75734c37946606125578aefa, user_id=001f4f2434334699ad83b86e54d9664e, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:41.146602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:41.160189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1358/2693] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:41 2022] PUT /identity/v3/projects/b788486b75734c37946606125578aefa/users/001f4f2434334699ad83b86e54d9664e/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:41.164213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:41.174179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68233988-95b3-4bab-8b02-f59f7f366e98 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:41.230521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab7e5-03dc-4c7e-b30b-9a93d4810517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:41.231136 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab7e5-03dc-4c7e-b30b-9a93d4810517 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.231423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab7e5-03dc-4c7e-b30b-9a93d4810517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.231701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab7e5-03dc-4c7e-b30b-9a93d4810517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.232385 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aab7e5-03dc-4c7e-b30b-9a93d4810517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:41.233999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aab7e5-03dc-4c7e-b30b-9a93d4810517 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:41.244160 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1336/2694] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:41.248217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:41.258373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085297-577f-41fc-8f06-9e9d66a98177 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:41.263200 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:41.263849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.264130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.264404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] PATH_INFO: `/v3/projects/b788486b75734c37946606125578aefa/users/001f4f2434334699ad83b86e54d9664e/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.266367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b788486b75734c37946606125578aefa, user_id=001f4f2434334699ad83b86e54d9664e, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:41.269003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:41.279480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1359/2695] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:41 2022] PUT /identity/v3/projects/b788486b75734c37946606125578aefa/users/001f4f2434334699ad83b86e54d9664e/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:41.289547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.289839 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.290121 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.317580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] MFA Rules not processed for user `001f4f2434334699ad83b86e54d9664e`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:41.318791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:41.386895 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1337/2696] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:41.397801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:41.399623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:41.456880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a5563-843d-46a2-b4f8-f7c6c2ebb47e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:41.460503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:41.461148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.461432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.461711 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.471544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:41.473012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:41.482898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1360/2697] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:41 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:41.528809 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:41.530176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:41.542769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a387a6-3f7b-435f-97b2-e70938018efb None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:41.546318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30bad8-34b7-4321-8d3f-48c7b80e22eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:41.546920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30bad8-34b7-4321-8d3f-48c7b80e22eb service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:41.547199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30bad8-34b7-4321-8d3f-48c7b80e22eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:41.547473 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30bad8-34b7-4321-8d3f-48c7b80e22eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:41.574359 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:41.606384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30bad8-34b7-4321-8d3f-48c7b80e22eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:41.607821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30bad8-34b7-4321-8d3f-48c7b80e22eb service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:41.616883 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1338/2698] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:44.893870 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:44.955432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ad67f-14d6-4aa5-a59b-93162071cef0 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:44.959890 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:44.960520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:44.960853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:44.961128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] PATH_INFO: `/v3/users/6866586e4aef44e78d2702a337cb97de` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:44.963202 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6866586e4aef44e78d2702a337cb97de)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:44.965222 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:45.008789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6866586e4aef44e78d2702a337cb97de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.016201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6866586e4aef44e78d2702a337cb97de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.018117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:45.030188 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6866586e4aef44e78d2702a337cb97de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.037146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6866586e4aef44e78d2702a337cb97de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.037765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6866586e4aef44e78d2702a337cb97de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.043606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1361/2699] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:03:44 2022] DELETE /identity/v3/users/6866586e4aef44e78d2702a337cb97de => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:45.048292 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:45.048850 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:45.048961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:45.069710 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:45.070926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:45.129302 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1339/2700] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:45.139340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:45.140792 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:45.197692 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430088c8-92b3-4e82-8082-2d70151ed915 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:45.201388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:45.202004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:45.202288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:45.202567 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:45.211442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:45.213337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:45.222658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1362/2701] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:45.491445 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:45.492926 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:45.505921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f522d0a-d345-4a24-bd1f-3c8f3926a88b None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:45.512252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:45.513114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:45.513471 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:45.513823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] PATH_INFO: `/v3/projects/4d32f6cf977e4836a6e8a8a2c9a36530` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:45.515557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d32f6cf977e4836a6e8a8a2c9a36530)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:45.517747 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:45.580370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d32f6cf977e4836a6e8a8a2c9a36530', 'request_id': 'req-b9a410d1-3f9d-4c54-8767-50cd172c51f7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.586672 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d32f6cf977e4836a6e8a8a2c9a36530', 'request_id': 'req-b9a410d1-3f9d-4c54-8767-50cd172c51f7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.594571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d32f6cf977e4836a6e8a8a2c9a36530', 'request_id': 'req-b9a410d1-3f9d-4c54-8767-50cd172c51f7'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.605441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] The token cache is being invalidate because project 4d32f6cf977e4836a6e8a8a2c9a36530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:03:45.605759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.606398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.607896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1340/2702] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:03:45 2022] DELETE /identity/v3/projects/4d32f6cf977e4836a6e8a8a2c9a36530 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:45.611811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:45.621434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fea35-b046-4ba4-a599-d88fba09a017 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:45.678515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:45.679087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:45.679385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:45.679656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] PATH_INFO: `/v3/users/001f4f2434334699ad83b86e54d9664e` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:45.680893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=001f4f2434334699ad83b86e54d9664e)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:45.682972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:45.731564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001f4f2434334699ad83b86e54d9664e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.738551 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001f4f2434334699ad83b86e54d9664e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.751455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001f4f2434334699ad83b86e54d9664e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.759116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001f4f2434334699ad83b86e54d9664e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.759756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7edd6af-cbc6-4995-bd10-9d15c62d8c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001f4f2434334699ad83b86e54d9664e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:45.761234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:45.767550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1363/2703] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:03:45 2022] DELETE /identity/v3/users/001f4f2434334699ad83b86e54d9664e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:46.037619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:46.039546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:46.053363 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a410d1-3f9d-4c54-8767-50cd172c51f7 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:46.103362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:46.115007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:46.115765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.116433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.117173 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] PATH_INFO: `/v3/projects/b788486b75734c37946606125578aefa` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.119457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b788486b75734c37946606125578aefa)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:46.121620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:46.169836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b788486b75734c37946606125578aefa', 'request_id': 'req-0c613362-71c1-4f99-a3b4-0520b040e16c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:46.176029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b788486b75734c37946606125578aefa', 'request_id': 'req-0c613362-71c1-4f99-a3b4-0520b040e16c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:46.184248 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b788486b75734c37946606125578aefa', 'request_id': 'req-0c613362-71c1-4f99-a3b4-0520b040e16c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:46.194669 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] The token cache is being invalidate because project b788486b75734c37946606125578aefa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:03:46.194957 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:46.195595 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-0c613362-71c1-4f99-a3b4-0520b040e16c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:46.197180 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1341/2704] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:03:46 2022] DELETE /identity/v3/projects/b788486b75734c37946606125578aefa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:46.202612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414f91c-a45e-4353-9931-b091cf7e191d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.203108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414f91c-a45e-4353-9931-b091cf7e191d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.203214 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414f91c-a45e-4353-9931-b091cf7e191d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.223634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9414f91c-a45e-4353-9931-b091cf7e191d None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:46.224787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9414f91c-a45e-4353-9931-b091cf7e191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:46.291119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1364/2705] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:46.299977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fad3e-2bbc-4479-a100-55b0a3611340 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.300264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fad3e-2bbc-4479-a100-55b0a3611340 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.300560 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fad3e-2bbc-4479-a100-55b0a3611340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.321133 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-727fad3e-2bbc-4479-a100-55b0a3611340 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:46.322761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-727fad3e-2bbc-4479-a100-55b0a3611340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:46.330789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1342/2706] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:46.335052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414f91c-a45e-4353-9931-b091cf7e191d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:46.345286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414f91c-a45e-4353-9931-b091cf7e191d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:46.350286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:46.350909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.351197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.351478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.352193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:03:46.352489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:46.354789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:46.367016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1365/2707] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:03:46.371546 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fad3e-2bbc-4479-a100-55b0a3611340 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:46.381000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fad3e-2bbc-4479-a100-55b0a3611340 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:46.386625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:46.387246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.387599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.387888 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.388559 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:46.390477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:46.418625 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1343/2708] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:03:46 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:46.422704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:46.434759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecde43-8859-4062-a7f1-4efd9dbabeca admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:46.494076 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:46.494701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.494997 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.495268 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.495984 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:46.497954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:46.522606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1366/2709] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:03:46 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:46.523976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:46.534976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693c2d8-a4bd-46a7-a4c0-981d717151d2 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:46.539733 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:46.540481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.540743 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.541025 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.541718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:46.543325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:46.553506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1344/2710] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:46.559041 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:46.571752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744716c-b5a6-4c5b-96e6-c340ecf53bab admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:46.577401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bd4ce-2337-455b-95dc-adc0d6c2c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:46.578810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bd4ce-2337-455b-95dc-adc0d6c2c3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.580284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bd4ce-2337-455b-95dc-adc0d6c2c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.580284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bd4ce-2337-455b-95dc-adc0d6c2c3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.580284 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bd4ce-2337-455b-95dc-adc0d6c2c3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:46.581710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bd4ce-2337-455b-95dc-adc0d6c2c3eb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:46.596014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1367/2711] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:46.611556 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:46.638859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b24997-e224-4e3e-a5df-26a85a19b0df admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:46.647040 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:46.661384 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.662728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.663843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] PATH_INFO: `/v3/projects/bfa39224f29645c89b20413f8cfc6e4c/users/4418a0689b7e43df98b7f47a9e3695ae/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.695759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa39224f29645c89b20413f8cfc6e4c, user_id=4418a0689b7e43df98b7f47a9e3695ae, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:46.711492 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:46.741974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1345/2712] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:46 2022] PUT /identity/v3/projects/bfa39224f29645c89b20413f8cfc6e4c/users/4418a0689b7e43df98b7f47a9e3695ae/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:46.753280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed26336-dedc-495d-a0a2-2069e697b857 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:46.753679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed26336-dedc-495d-a0a2-2069e697b857 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:46.754032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed26336-dedc-495d-a0a2-2069e697b857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:46.973318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2ed26336-dedc-495d-a0a2-2069e697b857 None None] MFA Rules not processed for user `4418a0689b7e43df98b7f47a9e3695ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:46.974722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2ed26336-dedc-495d-a0a2-2069e697b857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:47.051538 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:47.063416 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1368/2713] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:47.067914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.152424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c39f4-6abf-4d2d-97cb-647c26ce424a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.159468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.160213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.160645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.160960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.161800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.163921 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:47.197144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1346/2714] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:03:47 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:03:47.198919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed26336-dedc-495d-a0a2-2069e697b857 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.210295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed26336-dedc-495d-a0a2-2069e697b857 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.283543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.284171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.284665 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.284955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.285620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.287527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:47.311915 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1369/2715] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:03:47 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:47.316326 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.323010 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:47.330667 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e73e-a638-4b64-a56e-19b0e68e8261 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.338100 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.338841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.339189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.339583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.340375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.342054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:47.356681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1347/2716] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:47.358227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.371183 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa000da-a93b-4d78-9ba0-7b0b950dc1b4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.375947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.376683 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.376973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.377256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] PATH_INFO: `/v3/projects/9d6c5314a8d74b99b21bffda45c272a1/users/add021d6131a47c2826a75fbe82561c8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.394023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6c5314a8d74b99b21bffda45c272a1, user_id=add021d6131a47c2826a75fbe82561c8, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.397110 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:47.411586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1370/2717] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:47 2022] PUT /identity/v3/projects/9d6c5314a8d74b99b21bffda45c272a1/users/add021d6131a47c2826a75fbe82561c8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:47.415833 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.426872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d8227-3b67-4914-8fc6-2f33ba8d544c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.486408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37444b79-1220-46da-ac1c-db8a9d50aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.487045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37444b79-1220-46da-ac1c-db8a9d50aacf admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.487331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37444b79-1220-46da-ac1c-db8a9d50aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.487621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37444b79-1220-46da-ac1c-db8a9d50aacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.488312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37444b79-1220-46da-ac1c-db8a9d50aacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.489951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37444b79-1220-46da-ac1c-db8a9d50aacf admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:47.501021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1348/2718] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:03:47.504867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.515733 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2e75f-3b95-4d1f-af0a-b4307c631259 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.520564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.521253 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.521597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.521885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] PATH_INFO: `/v3/projects/9d6c5314a8d74b99b21bffda45c272a1/users/add021d6131a47c2826a75fbe82561c8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.523961 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6c5314a8d74b99b21bffda45c272a1, user_id=add021d6131a47c2826a75fbe82561c8, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.527154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:47.538318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1371/2719] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:47 2022] PUT /identity/v3/projects/9d6c5314a8d74b99b21bffda45c272a1/users/add021d6131a47c2826a75fbe82561c8/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:03:47.548683 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffc2ee-ee4a-4000-8bf2-ec5e8e66b07a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.549196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffc2ee-ee4a-4000-8bf2-ec5e8e66b07a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.549477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffc2ee-ee4a-4000-8bf2-ec5e8e66b07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.578990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2cffc2ee-ee4a-4000-8bf2-ec5e8e66b07a None None] MFA Rules not processed for user `add021d6131a47c2826a75fbe82561c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:47.580222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2cffc2ee-ee4a-4000-8bf2-ec5e8e66b07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:47.654147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1349/2720] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:47.666986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.668731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:47.739249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32370983-8df8-48a0-8549-4013070ad9ad admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.744450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.745291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.745574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.745849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.756929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.758489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:47.774510 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1372/2721] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:47.894147 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffc2ee-ee4a-4000-8bf2-ec5e8e66b07a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:47.906449 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffc2ee-ee4a-4000-8bf2-ec5e8e66b07a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:47.912366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d449a-1087-4de0-a4d2-e36330c59e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:47.913119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d449a-1087-4de0-a4d2-e36330c59e74 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:47.913474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d449a-1087-4de0-a4d2-e36330c59e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:47.913824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d449a-1087-4de0-a4d2-e36330c59e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:47.988086 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d449a-1087-4de0-a4d2-e36330c59e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:47.989565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d449a-1087-4de0-a4d2-e36330c59e74 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:48.000970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1350/2722] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:50.690128 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:50.743282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:50.760445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acf6a7-c1c1-4d5f-b195-99297a0991b6 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:50.765086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:50.765744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:50.766024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:50.766298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] PATH_INFO: `/v3/users/4418a0689b7e43df98b7f47a9e3695ae` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:50.767494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4418a0689b7e43df98b7f47a9e3695ae)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:50.769454 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:50.819658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4418a0689b7e43df98b7f47a9e3695ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:50.827936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4418a0689b7e43df98b7f47a9e3695ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:50.841726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4418a0689b7e43df98b7f47a9e3695ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:50.849818 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4418a0689b7e43df98b7f47a9e3695ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:50.850511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4418a0689b7e43df98b7f47a9e3695ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:50.857936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1373/2723] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:03:50 2022] DELETE /identity/v3/users/4418a0689b7e43df98b7f47a9e3695ae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:50.862497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:50.863023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:50.863224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:50.883941 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:50.885569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:50.891006 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=100 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:50.959150 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1351/2724] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:50.974326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:50.975834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:51.034755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763f1e8-ccd4-491a-929a-c6b26c3a5287 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:51.038489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:51.039109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:51.039394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:51.039675 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:51.048839 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:51.050266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:51.060465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1374/2725] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:51.324884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:51.326295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:51.341770 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a434-41d5-44a5-a244-e8186767d2df None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:51.346320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:51.346965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:51.347244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:51.347520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] PATH_INFO: `/v3/projects/bfa39224f29645c89b20413f8cfc6e4c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:51.348834 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa39224f29645c89b20413f8cfc6e4c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:51.350730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:51.398860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa39224f29645c89b20413f8cfc6e4c', 'request_id': 'req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.405243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa39224f29645c89b20413f8cfc6e4c', 'request_id': 'req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.412504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa39224f29645c89b20413f8cfc6e4c', 'request_id': 'req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.423859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] The token cache is being invalidate because project bfa39224f29645c89b20413f8cfc6e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:03:51.424156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.424828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.426469 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1352/2726] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:03:51 2022] DELETE /identity/v3/projects/bfa39224f29645c89b20413f8cfc6e4c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:51.430425 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:51.440875 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797315e-77a7-42be-9635-c94cc05d8937 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:51.495433 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:51.496062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:51.496340 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:51.496649 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] PATH_INFO: `/v3/users/add021d6131a47c2826a75fbe82561c8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:51.497847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add021d6131a47c2826a75fbe82561c8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:51.499678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:51.512880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:51.545413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add021d6131a47c2826a75fbe82561c8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.552394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add021d6131a47c2826a75fbe82561c8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.566057 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add021d6131a47c2826a75fbe82561c8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.574154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add021d6131a47c2826a75fbe82561c8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.574800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cd39f28b-fb5d-456b-8522-c2ea5eb24a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add021d6131a47c2826a75fbe82561c8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.581241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1375/2727] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:03:51 2022] DELETE /identity/v3/users/add021d6131a47c2826a75fbe82561c8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:51.850610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:51.852109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:51.867543 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585caa1f-a71b-4bbc-a39c-4c5dd87a4853 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:51.927305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:51.927981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:51.928233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:51.928508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] PATH_INFO: `/v3/projects/9d6c5314a8d74b99b21bffda45c272a1` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:51.929692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6c5314a8d74b99b21bffda45c272a1)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:51.931620 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:51.970444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:51.982536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6c5314a8d74b99b21bffda45c272a1', 'request_id': 'req-c4c0e19e-5428-4537-9842-c978f16afb2c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.988867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6c5314a8d74b99b21bffda45c272a1', 'request_id': 'req-c4c0e19e-5428-4537-9842-c978f16afb2c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:51.996732 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6c5314a8d74b99b21bffda45c272a1', 'request_id': 'req-c4c0e19e-5428-4537-9842-c978f16afb2c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:52.008138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] The token cache is being invalidate because project 9d6c5314a8d74b99b21bffda45c272a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:03:52.008428 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:52.009122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c4c0e19e-5428-4537-9842-c978f16afb2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:03:52.017540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1353/2728] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:03:51 2022] DELETE /identity/v3/projects/9d6c5314a8d74b99b21bffda45c272a1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:52.022620 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049ca86-1cdb-4abf-9b10-67d566d804bb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.023086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049ca86-1cdb-4abf-9b10-67d566d804bb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.023203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049ca86-1cdb-4abf-9b10-67d566d804bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.046557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7049ca86-1cdb-4abf-9b10-67d566d804bb None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:52.047681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7049ca86-1cdb-4abf-9b10-67d566d804bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:52.109972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1376/2729] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:52.119427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4200c-d8f3-4652-9304-f230813e2d08 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.119719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4200c-d8f3-4652-9304-f230813e2d08 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.120021 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4200c-d8f3-4652-9304-f230813e2d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.141200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-14c4200c-d8f3-4652-9304-f230813e2d08 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:52.142422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-14c4200c-d8f3-4652-9304-f230813e2d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:52.150882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1354/2730] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:52.155136 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049ca86-1cdb-4abf-9b10-67d566d804bb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:52.165843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049ca86-1cdb-4abf-9b10-67d566d804bb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:52.170524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:52.171143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.171428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.171706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.172393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:03:52.172734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:52.174349 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:52.186503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1377/2731] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:03:52.190796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4200c-d8f3-4652-9304-f230813e2d08 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:52.200731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4200c-d8f3-4652-9304-f230813e2d08 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:52.205342 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:52.205947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.206229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.206504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.207159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:52.209084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:52.236739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1355/2732] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:03:52 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:03:52.240787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:52.251198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54836f93-4698-43ec-a71c-ba2f1905e003 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:52.309901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:52.310524 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.310822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.311098 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.311755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:52.313679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:52.334786 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1378/2733] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:03:52 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:03:52.339873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:52.350616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe4e6c-ca91-4707-9f9a-b25de294f71c admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:52.355783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:52.356772 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.357156 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.357534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.358330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:52.360005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:52.373422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1356/2734] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:52.375039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:52.385867 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46022941-4f19-4494-8202-aadbaafc6ac7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:52.390513 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb762992-44db-496d-bcec-4e02904ce2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:52.391123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762992-44db-496d-bcec-4e02904ce2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.391403 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762992-44db-496d-bcec-4e02904ce2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.391677 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762992-44db-496d-bcec-4e02904ce2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.392357 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb762992-44db-496d-bcec-4e02904ce2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:52.393970 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb762992-44db-496d-bcec-4e02904ce2c4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:52.395363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:52.408085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1379/2735] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:03:52.412072 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:52.422607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5acc8-59f7-472d-b78d-18f2d38191cd admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:52.427336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:52.428001 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.428287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.428591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] PATH_INFO: `/v3/projects/07add2998d8a4be5918cda27e6118a65/users/9e4a2f1cbb4b4ff894007cb9c8730b52/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.443618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07add2998d8a4be5918cda27e6118a65, user_id=9e4a2f1cbb4b4ff894007cb9c8730b52, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:52.446923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:52.459826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1357/2736] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:03:52 2022] PUT /identity/v3/projects/07add2998d8a4be5918cda27e6118a65/users/9e4a2f1cbb4b4ff894007cb9c8730b52/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:03:52.471201 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338df071-c784-4dba-8ee2-625d31ca1dfc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.471489 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338df071-c784-4dba-8ee2-625d31ca1dfc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.471770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338df071-c784-4dba-8ee2-625d31ca1dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.499257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-338df071-c784-4dba-8ee2-625d31ca1dfc None None] MFA Rules not processed for user `9e4a2f1cbb4b4ff894007cb9c8730b52`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:52.500481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-338df071-c784-4dba-8ee2-625d31ca1dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:52.565755 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1380/2737] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:03:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:52.576414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:52.578269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:52.643493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4d16f-3243-44cb-b584-f4a49dc727d8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:52.647803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:52.650992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:52.650992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:52.650992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:52.662184 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:52.665059 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:52.682254 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1358/2738] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:03:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:03:53.283256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:53.283731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:53.283848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:53.302673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] MFA Rules not processed for user `a4141b562f1442efaa032122666724df`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:03:53.303865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:53.358490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1381/2739] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 29 07:03:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:03:53.399075 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:53.400530 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:03:53.468983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f76eb8-4c2e-4de2-b070-73239b44bc56 service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:03:53.474033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35b790a4a5b4306a5537984d0b449e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:03:53.474652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:03:53.474933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:03:53.475210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:03:53.480533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:03:53.484545 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:03:53.486719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:03:53.497186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1359/2740] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 29 07:03:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 07:03:59.984947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:03:59.986432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:00.059846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f97cb1-aae1-4996-8efa-924f126fe77e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:00.064337 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:00.065012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:00.065300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:00.065582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] PATH_INFO: `/v3/users/13af20e11c9449fd807eeeb1b3ff898f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:00.066760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] RBAC: Authorizing `identity:delete_user(user_id=13af20e11c9449fd807eeeb1b3ff898f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:00.068706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:00.103860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:00.116782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13af20e11c9449fd807eeeb1b3ff898f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.125111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13af20e11c9449fd807eeeb1b3ff898f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.137771 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13af20e11c9449fd807eeeb1b3ff898f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.145247 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13af20e11c9449fd807eeeb1b3ff898f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.145847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13af20e11c9449fd807eeeb1b3ff898f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.154210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1382/2741] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:03:59 2022] DELETE /identity/v3/users/13af20e11c9449fd807eeeb1b3ff898f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:04:00.174686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:00.185329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f6cbf-f519-4285-8efe-fa279ad64967 service swift] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:00.244988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:00.245644 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:00.245930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:00.246206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] PATH_INFO: `/v3/users/9e4a2f1cbb4b4ff894007cb9c8730b52` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:00.247378 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4a2f1cbb4b4ff894007cb9c8730b52)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:00.249323 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:00.296988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4a2f1cbb4b4ff894007cb9c8730b52'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.297645 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4a2f1cbb4b4ff894007cb9c8730b52'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.313821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4a2f1cbb4b4ff894007cb9c8730b52'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.321796 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4a2f1cbb4b4ff894007cb9c8730b52'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.323029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-e1a31374-95b1-4d41-8990-3120214112ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4a2f1cbb4b4ff894007cb9c8730b52'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.333570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:00.334995 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:00.348073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1360/2742] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:04:00 2022] DELETE /identity/v3/users/9e4a2f1cbb4b4ff894007cb9c8730b52 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:04:00.352980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:00.353263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:00.353538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:00.353761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da923e-f730-4b52-98e8-6c3055bf288f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:00.376183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:04:00.377435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:00.413827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:00.414474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:00.415406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:00.415689 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] PATH_INFO: `/v3/projects/387f03d497884045b998a6f788991e18` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:00.416901 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387f03d497884045b998a6f788991e18)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:00.418781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:00.437884 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1361/2743] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:04:00.449060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:00.450636 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:00.469730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:00.475394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387f03d497884045b998a6f788991e18', 'request_id': 'req-679730c2-abe6-4e35-9318-b2e6522ef632'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.489795 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387f03d497884045b998a6f788991e18', 'request_id': 'req-679730c2-abe6-4e35-9318-b2e6522ef632'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.498712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387f03d497884045b998a6f788991e18', 'request_id': 'req-679730c2-abe6-4e35-9318-b2e6522ef632'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.513000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] The token cache is being invalidate because project 387f03d497884045b998a6f788991e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:04:00.513318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.513989 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.516044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1383/2744] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:04:00 2022] DELETE /identity/v3/projects/387f03d497884045b998a6f788991e18 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:04:00.520582 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:00.524095 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:00.529604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59bbe0-f03e-4808-a6dd-4de72926e861 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:00.570614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:00.589611 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:00.590474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:00.590821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:00.591252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:00.593034 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:00.598249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679730c2-abe6-4e35-9318-b2e6522ef632 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:00.603835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:00.605119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:00.605410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:00.605693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] PATH_INFO: `/v3/users/19abe8b7770d47ffa9e8af720bd8f123` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:00.606749 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:00.607048 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19abe8b7770d47ffa9e8af720bd8f123)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:00.608774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:00.609058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:00.621019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1362/2745] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:04:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:04:00.663648 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19abe8b7770d47ffa9e8af720bd8f123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.673963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19abe8b7770d47ffa9e8af720bd8f123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.687153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19abe8b7770d47ffa9e8af720bd8f123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.694342 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19abe8b7770d47ffa9e8af720bd8f123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.694981 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19abe8b7770d47ffa9e8af720bd8f123'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:00.705174 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1384/2746] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:04:00 2022] DELETE /identity/v3/users/19abe8b7770d47ffa9e8af720bd8f123 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:04:00.883201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:00.893858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f6922-47f6-4491-b7f0-74f471f5d503 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:00.922523 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:00.924365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:00.962174 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:00.963091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:00.963375 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:00.963654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] PATH_INFO: `/v3/projects/a05af5175bb843b0ab3c0e78156c518d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:00.964864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05af5175bb843b0ab3c0e78156c518d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:00.966804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.006276 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f33b36-5330-413c-816e-e53e5e676357 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.014129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05af5175bb843b0ab3c0e78156c518d', 'request_id': 'req-62873099-8146-4eec-84ef-646b1abaad65'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.020567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05af5175bb843b0ab3c0e78156c518d', 'request_id': 'req-62873099-8146-4eec-84ef-646b1abaad65'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.029521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05af5175bb843b0ab3c0e78156c518d', 'request_id': 'req-62873099-8146-4eec-84ef-646b1abaad65'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.040657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] The token cache is being invalidate because project a05af5175bb843b0ab3c0e78156c518d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:04:01.040951 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.041630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-62873099-8146-4eec-84ef-646b1abaad65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.043176 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1363/2747] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:04:00 2022] DELETE /identity/v3/projects/a05af5175bb843b0ab3c0e78156c518d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:04:01.065788 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.066465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.066751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.067035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] PATH_INFO: `/v3/projects/07add2998d8a4be5918cda27e6118a65` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.068325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07add2998d8a4be5918cda27e6118a65)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.070375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.080435 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:01.126947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07add2998d8a4be5918cda27e6118a65', 'request_id': 'req-4eb36509-a516-4648-bbe0-98a0b9830176'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.136269 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07add2998d8a4be5918cda27e6118a65', 'request_id': 'req-4eb36509-a516-4648-bbe0-98a0b9830176'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.145384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07add2998d8a4be5918cda27e6118a65', 'request_id': 'req-4eb36509-a516-4648-bbe0-98a0b9830176'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.159145 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] The token cache is being invalidate because project 07add2998d8a4be5918cda27e6118a65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:04:01.159428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.160088 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4eb36509-a516-4648-bbe0-98a0b9830176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:01.161647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1385/2748] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:04:00 2022] DELETE /identity/v3/projects/07add2998d8a4be5918cda27e6118a65 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:04:01.167493 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d0b0f-de18-4d20-8dc9-4165ae4a214c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.167774 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d0b0f-de18-4d20-8dc9-4165ae4a214c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.168048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d0b0f-de18-4d20-8dc9-4165ae4a214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.176045 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:01.191758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-339d0b0f-de18-4d20-8dc9-4165ae4a214c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:04:01.193024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-339d0b0f-de18-4d20-8dc9-4165ae4a214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:01.258245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1364/2749] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:04:01.268519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369190c-cbc9-4a5b-9651-390613c973c9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.268909 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369190c-cbc9-4a5b-9651-390613c973c9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.269259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369190c-cbc9-4a5b-9651-390613c973c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.290698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d369190c-cbc9-4a5b-9651-390613c973c9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:04:01.292063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d369190c-cbc9-4a5b-9651-390613c973c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:01.300721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1386/2750] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:04:01.305087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d0b0f-de18-4d20-8dc9-4165ae4a214c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:01.315668 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d0b0f-de18-4d20-8dc9-4165ae4a214c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.320298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.321940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.322233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.322509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.323196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:04:01.323503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.325109 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.337087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1365/2751] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:04:01.341698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369190c-cbc9-4a5b-9651-390613c973c9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:01.351111 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369190c-cbc9-4a5b-9651-390613c973c9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.356360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.357011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.357294 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.357572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.358234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.360143 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.386873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1387/2752] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:04:01 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:04:01.391023 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:01.401459 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ff79-18e4-4aee-ade5-6991cfef1917 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.457590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.458203 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.458485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.458759 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.459408 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.461389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.484290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1366/2753] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:04:01 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:04:01.488326 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:01.500309 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198fcc-5386-41fa-a168-71c8dcd3537e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.505135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.505757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.506047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.506448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.507146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.509273 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.519579 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1388/2754] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:04:01.523547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:01.534980 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3e91d-1509-4629-bbe6-781036606344 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.539870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1f86-1aaf-4aee-9174-a5a391699bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.540630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1f86-1aaf-4aee-9174-a5a391699bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.540922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1f86-1aaf-4aee-9174-a5a391699bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.541202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1f86-1aaf-4aee-9174-a5a391699bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.541922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1f86-1aaf-4aee-9174-a5a391699bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.543497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1f86-1aaf-4aee-9174-a5a391699bf7 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.553717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1367/2755] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:04:01.558261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:01.568038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cece82c-1441-476c-aaa7-77a0468caba5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.573364 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.574042 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.574329 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.574610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] PATH_INFO: `/v3/projects/622c6d31b5ca414fb3e14f27190ec7f6/users/4a634f7de0c3486b978727801e62c0e7/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.589460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=622c6d31b5ca414fb3e14f27190ec7f6, user_id=4a634f7de0c3486b978727801e62c0e7, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.592037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.604532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1389/2756] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:04:01 2022] PUT /identity/v3/projects/622c6d31b5ca414fb3e14f27190ec7f6/users/4a634f7de0c3486b978727801e62c0e7/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:04:01.614583 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.614874 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.615153 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.644881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] MFA Rules not processed for user `4a634f7de0c3486b978727801e62c0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:04:01.646138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:01.706210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:01.712964 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1368/2757] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:04:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:04:01.724810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:01.726532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:01.784119 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f41988-4303-47e9-99f6-6a114b20d1b5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:01.787785 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4b8f7545934151aafa986da871b944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:01.788405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:01.788730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:01.789015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:01.798029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:01.799459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:01.808241 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1390/2758] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:04:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:04:08.513140 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:08.514640 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:08.583998 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9fba4-ada3-4545-b71e-2b3ce143441c None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:08.588379 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0e6e8-de70-4759-bdae-7af428514bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:08.589049 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0e6e8-de70-4759-bdae-7af428514bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:08.589330 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0e6e8-de70-4759-bdae-7af428514bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:08.589610 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0e6e8-de70-4759-bdae-7af428514bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:08.591811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f6b0e6e8-de70-4759-bdae-7af428514bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:08.602073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0e6e8-de70-4759-bdae-7af428514bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:08.603784 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0e6e8-de70-4759-bdae-7af428514bc5 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:08.613206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1369/2759] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:04:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:04:11.804256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:18.609414 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=27 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:26.397668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:26.459259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0425d-d8c5-46f1-9225-4346f75f8e1a service glance] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:26.464073 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:26.464732 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:26.465016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:26.465293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] PATH_INFO: `/v3/users/4a634f7de0c3486b978727801e62c0e7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:26.466436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a634f7de0c3486b978727801e62c0e7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:26.468330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:26.515298 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a634f7de0c3486b978727801e62c0e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:26.522085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a634f7de0c3486b978727801e62c0e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:26.533908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a634f7de0c3486b978727801e62c0e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:26.540596 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a634f7de0c3486b978727801e62c0e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:26.541325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a634f7de0c3486b978727801e62c0e7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:26.547373 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1391/2760] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:04:26 2022] DELETE /identity/v3/users/4a634f7de0c3486b978727801e62c0e7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:04:26.551875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:26.552390 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:26.552495 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:26.572792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:04:26.574135 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:26.633307 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1370/2761] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:04:26.643160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:26.644621 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:26.702972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ca25-0928-4af4-9ac6-3fbb9e732f41 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:26.706545 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:26.707153 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:26.707432 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:26.707704 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:26.718068 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:26.719481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:26.729922 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1392/2762] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:04:27.004696 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:04:27.006127 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:04:27.020276 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84530f31-4917-4ec2-9664-9517cd502fd9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:04:27.025659 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:04:27.026536 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:04:27.026829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:04:27.027114 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] PATH_INFO: `/v3/projects/622c6d31b5ca414fb3e14f27190ec7f6` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:04:27.028309 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=622c6d31b5ca414fb3e14f27190ec7f6)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:04:27.030282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:04:27.076189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '622c6d31b5ca414fb3e14f27190ec7f6', 'request_id': 'req-af45c9cd-07c4-411a-9e09-2fa938cbb887'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:27.082466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '622c6d31b5ca414fb3e14f27190ec7f6', 'request_id': 'req-af45c9cd-07c4-411a-9e09-2fa938cbb887'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:27.090923 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '622c6d31b5ca414fb3e14f27190ec7f6', 'request_id': 'req-af45c9cd-07c4-411a-9e09-2fa938cbb887'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:27.101426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] The token cache is being invalidate because project 622c6d31b5ca414fb3e14f27190ec7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:04:27.101716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:27.102362 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-af45c9cd-07c4-411a-9e09-2fa938cbb887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:04:27.104466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1371/2763] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:04:27 2022] DELETE /identity/v3/projects/622c6d31b5ca414fb3e14f27190ec7f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:04:36.724983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:04:37.098509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:05:10.068669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:05:10.070093 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:05:10.197930 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58040f-48f5-4f21-bfb0-b10851210a95 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:05:10.255747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e65ef9-6dcf-427b-a12d-22b9e33f4b09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:05:10.256359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65ef9-6dcf-427b-a12d-22b9e33f4b09 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:05:10.256671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65ef9-6dcf-427b-a12d-22b9e33f4b09 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:05:10.256952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65ef9-6dcf-427b-a12d-22b9e33f4b09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:05:10.258680 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-50e65ef9-6dcf-427b-a12d-22b9e33f4b09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:05:10.269882 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e65ef9-6dcf-427b-a12d-22b9e33f4b09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:05:10.271441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e65ef9-6dcf-427b-a12d-22b9e33f4b09 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:05:10.281973 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1393/2764] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:05:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:05:16.275168 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:05:16.275767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:05:16.275767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:05:16.294990 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] MFA Rules not processed for user `f0f388a7fdbc4925940a5e1a53f15f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:05:16.296189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:05:16.360945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1372/2765] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:05:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:05:20.277859 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=84 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:05:26.352769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=25 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:07:19.983562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:19.984126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:19.984126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:20.003707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] MFA Rules not processed for user `f0f388a7fdbc4925940a5e1a53f15f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:07:20.004923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:20.011723 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1394/2766] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:07:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:07:20.328626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:07:20.330748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:20.388425 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7a70d-8dee-4005-bdba-756bc5b2a78a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:07:20.391993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:07:20.392704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:20.392987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:20.393258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:20.394993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:20.404354 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:07:20.405811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:07:20.413860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1373/2767] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:07:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:07:20.866904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:07:20.868484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:20.884726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28ea89-fa12-4805-a705-c350620506ef None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:07:20.889667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:07:20.890315 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:20.890594 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:20.890864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:20.893308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:20.905472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:07:20.907054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:07:20.917458 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1395/2768] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:07:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:07:22.296056 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:07:22.297697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:22.358608 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94cd33-8d00-42c3-83c0-c3cb306e1c37 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:07:22.364423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:07:22.365091 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:22.365376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:22.365653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:22.367415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:22.379771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:07:22.381260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:07:22.390899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1374/2769] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:07:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:07:30.241178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:07:30.242612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:30.263852 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b200dc9-9ffe-4e32-8d86-da37b95db29f service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:07:30.267644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:07:30.268236 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:30.268573 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:30.268851 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:30.270564 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:30.340260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:07:30.341764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:07:30.352659 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1396/2770] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:07:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:07:31.530569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:07:31.540481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b1285-dab5-4f1a-a0e1-967296ab811e service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:07:31.544050 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:07:31.544692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:31.544979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:31.545264 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:31.547017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:31.562582 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:07:31.563992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:07:31.574804 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1375/2771] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:07:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:07:34.421493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:07:34.424062 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:34.439935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ed888-c391-42d0-a613-6319b1ef01ee service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:07:34.445390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:07:34.446005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:34.446283 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:34.446557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:34.448783 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:34.461540 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:07:34.463409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:07:34.475103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1397/2772] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:07:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:07:38.341041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:07:38.354958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3aba4-6820-4151-9aa6-0170d2da57cf service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:07:38.361603 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:07:38.362247 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:07:38.362534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:07:38.362814 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:07:38.364641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:07:38.378105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:07:38.379803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:07:38.392994 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1376/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:07:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:07:44.470286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=53 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:07:48.387508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=64 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:08:36.482663 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:08:36.484061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:08:36.528172 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347594e-c28b-430a-ac1b-5f793a89c0fc service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:08:36.532529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:08:36.533163 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:08:36.533447 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:08:36.533722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:08:36.535479 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:08:36.615716 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:08:36.617168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:08:36.636306 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1398/2774] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:08:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:08:46.633685 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=55 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:09:10.279717 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:10.299976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da67dc-58b7-407a-af49-6e4930bf3e69 service nova] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:10.304728 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:10.305489 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:10.305740 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:10.305946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:10.307935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:09:10.321032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:10.322404 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:10.333727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1377/2775] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:09:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:09:18.917429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:18.940450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c28ee6-7318-4bca-afcc-4a4123613157 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:18.947168 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe64416672b408f8cdf93220cc31ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:18.947807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:18.948096 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:18.948379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:18.950469 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:09:18.963811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:18.965423 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:18.986738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1399/2776] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:09:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:09:20.330060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=11 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:09:22.694305 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:22.703706 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99181523-4d16-404c-8c1a-1f5b5e9d4f7f service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:22.707949 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:22.708701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:22.708993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:22.709313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:22.711521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:09:22.777528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:22.779005 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:22.791396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1378/2777] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:09:25.582587 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:25.595539 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ecc4a-89fb-4302-8df3-fc29f5a13046 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:25.601417 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:25.602194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:25.602586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:25.602878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:25.605467 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:09:25.620431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:25.621990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:25.635856 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1400/2778] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:09:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:09:28.739192 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:28.741265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:09:28.756983 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156fc53-25c5-4b77-8c72-fb8b1a150590 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:28.762758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:28.763424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:28.763880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:28.764165 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] PATH_INFO: `/v3/users/f0f388a7fdbc4925940a5e1a53f15f9e` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:28.765447 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f388a7fdbc4925940a5e1a53f15f9e)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:28.767938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:28.829151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f388a7fdbc4925940a5e1a53f15f9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:28.829971 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f388a7fdbc4925940a5e1a53f15f9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:28.842734 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f388a7fdbc4925940a5e1a53f15f9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:28.853365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f388a7fdbc4925940a5e1a53f15f9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:28.854104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f388a7fdbc4925940a5e1a53f15f9e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:28.869523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1379/2779] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:09:28 2022] DELETE /identity/v3/users/f0f388a7fdbc4925940a5e1a53f15f9e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:09:29.048813 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:29.060385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200c176-1ea1-43f8-9c57-7d810c9e57c0 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:29.122754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:29.123387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:29.123667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:29.123942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] PATH_INFO: `/v3/projects/17de3e5c46bb43a3973d8b9728c797b7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:29.125102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17de3e5c46bb43a3973d8b9728c797b7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:29.126986 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:29.171450 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17de3e5c46bb43a3973d8b9728c797b7', 'request_id': 'req-4da03509-94d6-4e03-965d-61a493a221a3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.179504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17de3e5c46bb43a3973d8b9728c797b7', 'request_id': 'req-4da03509-94d6-4e03-965d-61a493a221a3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.185577 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17de3e5c46bb43a3973d8b9728c797b7', 'request_id': 'req-4da03509-94d6-4e03-965d-61a493a221a3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.196173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] The token cache is being invalidate because project 17de3e5c46bb43a3973d8b9728c797b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:09:29.196509 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.197171 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.198669 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1401/2780] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:09:29 2022] DELETE /identity/v3/projects/17de3e5c46bb43a3973d8b9728c797b7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:09:29.202826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:29.204260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:09:29.217529 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bb40a-94f2-44de-8712-f573686ebbc8 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:29.275314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:29.275879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:29.276152 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:29.276430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] PATH_INFO: `/v3/users/18450b80ca4b44a5b3db1babbdc0baef` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:29.277627 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18450b80ca4b44a5b3db1babbdc0baef)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:29.279524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:29.285429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:09:29.326481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18450b80ca4b44a5b3db1babbdc0baef'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.327130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18450b80ca4b44a5b3db1babbdc0baef'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.339535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18450b80ca4b44a5b3db1babbdc0baef'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.346938 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18450b80ca4b44a5b3db1babbdc0baef'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.347572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18450b80ca4b44a5b3db1babbdc0baef'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.360821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1380/2781] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:09:29 2022] DELETE /identity/v3/users/18450b80ca4b44a5b3db1babbdc0baef => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:09:29.533030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:09:29.534706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:09:29.550938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03509-94d6-4e03-965d-61a493a221a3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:09:29.611612 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:09:29.612259 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:09:29.612634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:09:29.612840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] PATH_INFO: `/v3/projects/f14ec3dd113b4c3e842abdf9a3c4fcf1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:09:29.614007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14ec3dd113b4c3e842abdf9a3c4fcf1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:09:29.615923 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:09:29.622107 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:09:29.664092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14ec3dd113b4c3e842abdf9a3c4fcf1', 'request_id': 'req-dabf8331-9477-4430-968b-705ae6a8f8dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.671297 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14ec3dd113b4c3e842abdf9a3c4fcf1', 'request_id': 'req-dabf8331-9477-4430-968b-705ae6a8f8dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.677757 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14ec3dd113b4c3e842abdf9a3c4fcf1', 'request_id': 'req-dabf8331-9477-4430-968b-705ae6a8f8dd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.688572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] The token cache is being invalidate because project f14ec3dd113b4c3e842abdf9a3c4fcf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:09:29.688878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.689583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:09:29.691166 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1402/2782] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:09:29 2022] DELETE /identity/v3/projects/f14ec3dd113b4c3e842abdf9a3c4fcf1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:09:39.356580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:09:39.684990 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:15.726928 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:15.728431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:15.799538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb870-db2c-42c3-9509-cdfa0678523f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:15.857671 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:15.858505 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:15.858787 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:15.859063 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:15.860832 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:15.871673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:15.873261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:15.883035 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1381/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:10:25.878826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=58 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:35.230400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:35.231805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:35.289362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf8331-9477-4430-968b-705ae6a8f8dd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:35.292964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:35.293578 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:35.293865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:35.294146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:35.295884 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:35.357282 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:35.358643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:35.368359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1403/2784] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:10:38.411482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:38.412906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:38.425769 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465935-6634-4161-9176-600255a5b5ea service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:38.430873 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:38.431485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:38.431826 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:38.432105 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:38.433821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:38.449970 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:38.452822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:38.462893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:38.464764 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:38.474878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1382/2785] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:10:38.486461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224edba-141f-4cc4-9e40-1cea9a252767 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:38.491245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:38.491891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:38.492182 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:38.492461 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] PATH_INFO: `/v3/users/217b321936224e2ba928b983c4e40017` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:38.493713 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217b321936224e2ba928b983c4e40017)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:38.495656 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:38.542807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217b321936224e2ba928b983c4e40017'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.550544 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217b321936224e2ba928b983c4e40017'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.563100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217b321936224e2ba928b983c4e40017'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.570400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217b321936224e2ba928b983c4e40017'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.571017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-076b4344-ceac-4cc5-8ded-841fe616cbe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217b321936224e2ba928b983c4e40017'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.577053 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1404/2786] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:10:38 2022] DELETE /identity/v3/users/217b321936224e2ba928b983c4e40017 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:10:38.760462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:38.773682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61cf94-e329-4a04-a57d-682175003061 service placement] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:38.837681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:38.838332 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:38.838618 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:38.838897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] PATH_INFO: `/v3/projects/f8d1f62a52884ebb91749ef49de11bec` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:38.840052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d1f62a52884ebb91749ef49de11bec)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:38.842857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:38.888003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d1f62a52884ebb91749ef49de11bec', 'request_id': 'req-808d13fb-38be-419d-9651-bfd079c50910'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.894031 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d1f62a52884ebb91749ef49de11bec', 'request_id': 'req-808d13fb-38be-419d-9651-bfd079c50910'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.901008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d1f62a52884ebb91749ef49de11bec', 'request_id': 'req-808d13fb-38be-419d-9651-bfd079c50910'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.911766 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] The token cache is being invalidate because project f8d1f62a52884ebb91749ef49de11bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:10:38.912057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.912723 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-808d13fb-38be-419d-9651-bfd079c50910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:38.914225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1383/2787] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:10:38 2022] DELETE /identity/v3/projects/f8d1f62a52884ebb91749ef49de11bec => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:38.921242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803a23d-9bb6-4880-b3c3-a84d39ef8d95 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:38.921529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803a23d-9bb6-4880-b3c3-a84d39ef8d95 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:38.921807 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803a23d-9bb6-4880-b3c3-a84d39ef8d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:38.941133 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e803a23d-9bb6-4880-b3c3-a84d39ef8d95 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:38.942481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e803a23d-9bb6-4880-b3c3-a84d39ef8d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:38.959752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:39.004769 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1405/2788] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:39.014137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7b64d-6bf1-485a-9e4c-8f1889871406 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.014421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7b64d-6bf1-485a-9e4c-8f1889871406 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.014701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7b64d-6bf1-485a-9e4c-8f1889871406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.034810 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-38c7b64d-6bf1-485a-9e4c-8f1889871406 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:39.036016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-38c7b64d-6bf1-485a-9e4c-8f1889871406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:39.044009 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1384/2789] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:39.048202 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803a23d-9bb6-4880-b3c3-a84d39ef8d95 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:39.057699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803a23d-9bb6-4880-b3c3-a84d39ef8d95 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:39.062271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:39.062869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.063147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.063422 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.064102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:10:39.064403 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:39.066016 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:39.077731 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1406/2790] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:10:39.081979 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7b64d-6bf1-485a-9e4c-8f1889871406 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:39.092232 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7b64d-6bf1-485a-9e4c-8f1889871406 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:39.096968 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:39.097573 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.097858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.098138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.098812 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:39.100750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:39.129275 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1385/2791] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:10:39 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:10:39.133392 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:39.143390 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7862b-9e55-4503-87e4-1d2a6cfa8e12 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:39.198610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:39.199245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.199526 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.199800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.200451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:39.202558 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:39.223585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1407/2792] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:10:39 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:10:39.227580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:39.238329 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4960ca-5f9f-42ae-85be-3b7ae9c28a8a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:39.242965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:39.243570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.243854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.244134 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.244861 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:39.246547 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:39.257624 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1386/2793] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:10:39.261480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:39.271394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b0dab-2f20-4e06-b64e-4de8bd544db6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:39.276285 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf365fb6-2934-474e-8602-500727a5a8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:39.276933 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf365fb6-2934-474e-8602-500727a5a8fd admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.277223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf365fb6-2934-474e-8602-500727a5a8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.277506 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf365fb6-2934-474e-8602-500727a5a8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.278193 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf365fb6-2934-474e-8602-500727a5a8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:39.280000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf365fb6-2934-474e-8602-500727a5a8fd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:39.290496 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1408/2794] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:10:39.295024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:39.305206 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce47d7-c83f-466b-a4b9-078bbb7a2002 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:39.309864 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:39.311223 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.311509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.311792 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] PATH_INFO: `/v3/projects/538582dfcca54f6386e69707d57f3bcf/users/b70d791374fd44e48ba405e2e9fe0e74/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.326062 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538582dfcca54f6386e69707d57f3bcf, user_id=b70d791374fd44e48ba405e2e9fe0e74, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:39.328746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:39.342906 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1387/2795] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:10:39 2022] PUT /identity/v3/projects/538582dfcca54f6386e69707d57f3bcf/users/b70d791374fd44e48ba405e2e9fe0e74/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:39.353240 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61fcd-b768-4f14-ad0c-009f9ab73449 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.353528 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61fcd-b768-4f14-ad0c-009f9ab73449 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.353801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61fcd-b768-4f14-ad0c-009f9ab73449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.381633 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-48c61fcd-b768-4f14-ad0c-009f9ab73449 None None] MFA Rules not processed for user `b70d791374fd44e48ba405e2e9fe0e74`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:39.382864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-48c61fcd-b768-4f14-ad0c-009f9ab73449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:39.445676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1409/2796] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:39.457028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:39.458768 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:39.474261 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:39.518927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e964d7a-ed67-4343-a709-a2b2aef9a2d4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:39.522533 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd56ed5-4285-42a8-9e9f-1d93184ac951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:39.523159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd56ed5-4285-42a8-9e9f-1d93184ac951 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:39.523444 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd56ed5-4285-42a8-9e9f-1d93184ac951 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:39.523724 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd56ed5-4285-42a8-9e9f-1d93184ac951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:39.532748 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd56ed5-4285-42a8-9e9f-1d93184ac951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:39.534179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd56ed5-4285-42a8-9e9f-1d93184ac951 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:39.543007 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1388/2797] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:10:48.292038 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c61fcd-b768-4f14-ad0c-009f9ab73449 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:48.311827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:48.355725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c61fcd-b768-4f14-ad0c-009f9ab73449 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:48.361091 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:48.361729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:48.362015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:48.362293 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] PATH_INFO: `/v3/users/b70d791374fd44e48ba405e2e9fe0e74` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:48.363443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70d791374fd44e48ba405e2e9fe0e74)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:48.365385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:48.412555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70d791374fd44e48ba405e2e9fe0e74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.419521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70d791374fd44e48ba405e2e9fe0e74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.431272 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70d791374fd44e48ba405e2e9fe0e74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.439657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70d791374fd44e48ba405e2e9fe0e74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.440277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70d791374fd44e48ba405e2e9fe0e74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.446343 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1410/2798] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:10:48 2022] DELETE /identity/v3/users/b70d791374fd44e48ba405e2e9fe0e74 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:48.450829 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:48.451612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:48.451612 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:48.471189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:48.472777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:48.531253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1389/2799] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:48.541010 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:48.542474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:48.600768 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca679ca8-9de5-4211-b229-6c965cbdb518 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:48.604441 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60dfd9-4dbe-4e99-b7b4-4b024655eabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:48.605085 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60dfd9-4dbe-4e99-b7b4-4b024655eabc service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:48.605363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60dfd9-4dbe-4e99-b7b4-4b024655eabc service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:48.605634 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60dfd9-4dbe-4e99-b7b4-4b024655eabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:48.616014 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60dfd9-4dbe-4e99-b7b4-4b024655eabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:48.617738 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60dfd9-4dbe-4e99-b7b4-4b024655eabc service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:48.628728 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1411/2800] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:10:48.897791 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:48.899240 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:48.913382 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433339a-4fb1-4b68-8c20-a0f9ef07ca04 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:48.918230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:48.918880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:48.919161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:48.919438 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] PATH_INFO: `/v3/projects/538582dfcca54f6386e69707d57f3bcf` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:48.920822 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=538582dfcca54f6386e69707d57f3bcf)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:48.922727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:48.970217 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538582dfcca54f6386e69707d57f3bcf', 'request_id': 'req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.976256 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538582dfcca54f6386e69707d57f3bcf', 'request_id': 'req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.984068 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538582dfcca54f6386e69707d57f3bcf', 'request_id': 'req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.994596 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] The token cache is being invalidate because project 538582dfcca54f6386e69707d57f3bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:10:48.994886 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.995508 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-7dbef1e4-3269-4cf1-bf9f-0c3a4e11e96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:48.997054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1390/2801] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:10:48 2022] DELETE /identity/v3/projects/538582dfcca54f6386e69707d57f3bcf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:49.002061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708867-5af4-49d6-9a30-81d6ace076f2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.002350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708867-5af4-49d6-9a30-81d6ace076f2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.002623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708867-5af4-49d6-9a30-81d6ace076f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.022321 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a8708867-5af4-49d6-9a30-81d6ace076f2 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:49.023503 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a8708867-5af4-49d6-9a30-81d6ace076f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:49.056217 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:49.086954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1412/2802] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:49.095746 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45598c-d608-4a58-a734-16f5da51a6b4 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.096041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45598c-d608-4a58-a734-16f5da51a6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.096324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45598c-d608-4a58-a734-16f5da51a6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.115821 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-4c45598c-d608-4a58-a734-16f5da51a6b4 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:49.117069 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-4c45598c-d608-4a58-a734-16f5da51a6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:49.125626 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1391/2803] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:49.129725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708867-5af4-49d6-9a30-81d6ace076f2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.139954 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708867-5af4-49d6-9a30-81d6ace076f2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.144574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.145122 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.145400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.145678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.146353 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:10:49.146653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.148216 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.159385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1413/2804] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:10:49.163509 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45598c-d608-4a58-a734-16f5da51a6b4 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.172575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45598c-d608-4a58-a734-16f5da51a6b4 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.177194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.177790 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.178067 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.178337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.178981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.180887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.208701 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1392/2805] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:10:49.213087 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.223235 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5970-e80c-4097-a9a7-a5170aad1985 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.279742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.280362 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.280672 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.280950 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.281613 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.283499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.303854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1414/2806] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:10:49.308260 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.319197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31898d-cbb6-4440-9279-629c6a2baecc admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.323789 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.324396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.324718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.324999 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.325688 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.327858 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.333191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:49.342365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1393/2807] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:10:49.343983 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.354219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0457-6462-4a14-983e-a74d46bc0f0f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.359035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c97ec-a017-46f7-b431-bd56f4159a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.359635 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c97ec-a017-46f7-b431-bd56f4159a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.359916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c97ec-a017-46f7-b431-bd56f4159a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.360190 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c97ec-a017-46f7-b431-bd56f4159a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.360908 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c97ec-a017-46f7-b431-bd56f4159a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.362483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c97ec-a017-46f7-b431-bd56f4159a4d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.373735 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1415/2808] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:10:49.378209 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.388088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4335a-8658-481a-98cb-596d6b0d20f5 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.393929 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.394648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.394867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.395144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] PATH_INFO: `/v3/projects/977eed0c25a34f75900509f2f2247489/users/b749a573d9814274b610d0146fea60f2/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.410057 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=977eed0c25a34f75900509f2f2247489, user_id=b749a573d9814274b610d0146fea60f2, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.412739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.431356 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1394/2809] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:10:49 2022] PUT /identity/v3/projects/977eed0c25a34f75900509f2f2247489/users/b749a573d9814274b610d0146fea60f2/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:49.443652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a3ec8-a9e5-4ed4-a843-679542f4167e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.443942 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a3ec8-a9e5-4ed4-a843-679542f4167e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.444218 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a3ec8-a9e5-4ed4-a843-679542f4167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.472030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-676a3ec8-a9e5-4ed4-a843-679542f4167e None None] MFA Rules not processed for user `b749a573d9814274b610d0146fea60f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:49.473258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-676a3ec8-a9e5-4ed4-a843-679542f4167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:49.540806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1416/2810] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:49.545806 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.608905 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e008de-1ef5-4295-929e-9593278c95e9 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.613652 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.614274 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.614561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.614841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.615515 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.617486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.644245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1395/2811] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:10:49.648459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a3ec8-a9e5-4ed4-a843-679542f4167e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.659375 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a3ec8-a9e5-4ed4-a843-679542f4167e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.691966 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:49.718471 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.719165 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.719457 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.719736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.720388 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.722330 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.747765 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1417/2812] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:10:49.749599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.760231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6127c2-4f56-431f-9486-4e7ecdab9d58 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.765084 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.765685 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.765967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.766239 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.766945 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.768518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.779048 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1396/2813] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:10:49.783610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.794019 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b73a8-f1da-4e44-bdf6-d20e707427f4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.799815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.800486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.800964 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.801501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] PATH_INFO: `/v3/projects/b4c7b38b44bb4fea80fd7d5f3c3237fd/users/6b10c282fb1a4273b0b5825f8e21bd64/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.818705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c7b38b44bb4fea80fd7d5f3c3237fd, user_id=6b10c282fb1a4273b0b5825f8e21bd64, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.821434 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.834698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1418/2814] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:10:49 2022] PUT /identity/v3/projects/b4c7b38b44bb4fea80fd7d5f3c3237fd/users/6b10c282fb1a4273b0b5825f8e21bd64/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:49.838962 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.849762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9941b-570b-4f41-b896-b754ac390727 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.906313 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389d0c8-3451-4641-a71d-c6ccd795a92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.906852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389d0c8-3451-4641-a71d-c6ccd795a92f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.907137 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389d0c8-3451-4641-a71d-c6ccd795a92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.907416 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389d0c8-3451-4641-a71d-c6ccd795a92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.908098 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389d0c8-3451-4641-a71d-c6ccd795a92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.909725 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389d0c8-3451-4641-a71d-c6ccd795a92f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.919381 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1397/2815] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:10:49.924464 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:49.935064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1319621-ea83-48c1-8d2a-33c311c3dc45 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:49.939744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:49.940401 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.940721 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.941004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] PATH_INFO: `/v3/projects/b4c7b38b44bb4fea80fd7d5f3c3237fd/users/6b10c282fb1a4273b0b5825f8e21bd64/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.943009 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c7b38b44bb4fea80fd7d5f3c3237fd, user_id=6b10c282fb1a4273b0b5825f8e21bd64, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:49.945706 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:49.956037 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1419/2816] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:10:49 2022] PUT /identity/v3/projects/b4c7b38b44bb4fea80fd7d5f3c3237fd/users/6b10c282fb1a4273b0b5825f8e21bd64/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:49.968249 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bb603-570d-459d-90d2-4cc7c2996ee9 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:49.968564 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bb603-570d-459d-90d2-4cc7c2996ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:49.968857 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bb603-570d-459d-90d2-4cc7c2996ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:49.996793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2e9bb603-570d-459d-90d2-4cc7c2996ee9 None None] MFA Rules not processed for user `6b10c282fb1a4273b0b5825f8e21bd64`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:49.998030 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2e9bb603-570d-459d-90d2-4cc7c2996ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:50.030423 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:50.066089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1398/2817] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:49 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:50.077679 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:50.079410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:50.136066 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0838251-3645-4c8a-9944-f948a5677e09 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:50.140026 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:50.140673 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:50.140963 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:50.141244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:50.150249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:50.151682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:50.161658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1420/2818] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:50 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:10:50.235647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bb603-570d-459d-90d2-4cc7c2996ee9 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:50.245280 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bb603-570d-459d-90d2-4cc7c2996ee9 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:50.248913 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f605e-b10b-4222-bffb-d9f1b6b4369a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:50.249514 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f605e-b10b-4222-bffb-d9f1b6b4369a service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:50.249795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f605e-b10b-4222-bffb-d9f1b6b4369a service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:50.250070 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f605e-b10b-4222-bffb-d9f1b6b4369a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:50.306430 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f605e-b10b-4222-bffb-d9f1b6b4369a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:50.307843 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f605e-b10b-4222-bffb-d9f1b6b4369a service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:50.316563 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1399/2819] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:50 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:10:59.429676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:59.490994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50d7d2-3618-4805-96cd-a7937bec6adc service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:59.495494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:59.496121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:59.496399 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:59.496699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] PATH_INFO: `/v3/users/b749a573d9814274b610d0146fea60f2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:59.497849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b749a573d9814274b610d0146fea60f2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:59.499895 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:59.533263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:10:59.546379 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b749a573d9814274b610d0146fea60f2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:59.553384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b749a573d9814274b610d0146fea60f2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:59.565029 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b749a573d9814274b610d0146fea60f2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:59.572210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b749a573d9814274b610d0146fea60f2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:59.572869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b749a573d9814274b610d0146fea60f2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:10:59.579335 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1421/2820] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:10:59 2022] DELETE /identity/v3/users/b749a573d9814274b610d0146fea60f2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:10:59.583852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:59.584336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:59.584435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:59.603862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:10:59.605082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:59.663763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1400/2821] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:10:59.673712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:10:59.675146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:10:59.734691 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1caecc-7b95-4427-9bd1-de4c8d5a5e61 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:10:59.738351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:10:59.738965 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:10:59.739244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:10:59.739517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:10:59.749007 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:10:59.750420 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:10:59.759703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1422/2822] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:00.031094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:00.032638 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:00.047318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03c5ed-76f9-4114-a975-42c216d405c6 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:00.052763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:00.053412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.054357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.054647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] PATH_INFO: `/v3/projects/977eed0c25a34f75900509f2f2247489` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.056008 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=977eed0c25a34f75900509f2f2247489)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:00.057939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:00.105282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977eed0c25a34f75900509f2f2247489', 'request_id': 'req-8563167c-8128-43c1-b9c2-62161b2ab7b4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.112461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977eed0c25a34f75900509f2f2247489', 'request_id': 'req-8563167c-8128-43c1-b9c2-62161b2ab7b4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.118963 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977eed0c25a34f75900509f2f2247489', 'request_id': 'req-8563167c-8128-43c1-b9c2-62161b2ab7b4'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.122865 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:00.130933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] The token cache is being invalidate because project 977eed0c25a34f75900509f2f2247489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:00.131229 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.132188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.133881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1401/2823] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:00 2022] DELETE /identity/v3/projects/977eed0c25a34f75900509f2f2247489 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:00.137948 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:00.148102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ffe64-1090-4601-bb49-7039480bbbb9 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:00.203384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:00.204018 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.204302 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.204619 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] PATH_INFO: `/v3/users/6b10c282fb1a4273b0b5825f8e21bd64` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.205812 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b10c282fb1a4273b0b5825f8e21bd64)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:00.207722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:00.253387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b10c282fb1a4273b0b5825f8e21bd64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.258647 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:00.260941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b10c282fb1a4273b0b5825f8e21bd64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.273941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b10c282fb1a4273b0b5825f8e21bd64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.281367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b10c282fb1a4273b0b5825f8e21bd64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.282055 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-81b84338-4185-4019-8ca8-f611772153f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b10c282fb1a4273b0b5825f8e21bd64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.287854 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1423/2824] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:00 2022] DELETE /identity/v3/users/6b10c282fb1a4273b0b5825f8e21bd64 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:00.556846 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:00.558643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:00.572412 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563167c-8128-43c1-b9c2-62161b2ab7b4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:00.631018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:00.631718 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.631946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.632222 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] PATH_INFO: `/v3/projects/b4c7b38b44bb4fea80fd7d5f3c3237fd` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.633397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c7b38b44bb4fea80fd7d5f3c3237fd)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:00.635315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:00.680807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c7b38b44bb4fea80fd7d5f3c3237fd', 'request_id': 'req-b8d51342-f1a6-45e7-ad2a-4053803e066c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.687490 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c7b38b44bb4fea80fd7d5f3c3237fd', 'request_id': 'req-b8d51342-f1a6-45e7-ad2a-4053803e066c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.694745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c7b38b44bb4fea80fd7d5f3c3237fd', 'request_id': 'req-b8d51342-f1a6-45e7-ad2a-4053803e066c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.705656 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] The token cache is being invalidate because project b4c7b38b44bb4fea80fd7d5f3c3237fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:00.705965 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.706581 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-b8d51342-f1a6-45e7-ad2a-4053803e066c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:00.708060 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1402/2825] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:00 2022] DELETE /identity/v3/projects/b4c7b38b44bb4fea80fd7d5f3c3237fd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:11:00.713580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ecc0b-b481-4776-aae4-a7575fc5ab60 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.714024 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ecc0b-b481-4776-aae4-a7575fc5ab60 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.714134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ecc0b-b481-4776-aae4-a7575fc5ab60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.734773 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-248ecc0b-b481-4776-aae4-a7575fc5ab60 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:00.736012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-248ecc0b-b481-4776-aae4-a7575fc5ab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:00.795139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1424/2826] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:00.804532 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903209a-30ab-4e7b-8f42-2ac7f958d9f5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.804856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903209a-30ab-4e7b-8f42-2ac7f958d9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.805138 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903209a-30ab-4e7b-8f42-2ac7f958d9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.825231 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-2903209a-30ab-4e7b-8f42-2ac7f958d9f5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:00.826441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-2903209a-30ab-4e7b-8f42-2ac7f958d9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:00.834417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1403/2827] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:00.838791 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ecc0b-b481-4776-aae4-a7575fc5ab60 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:00.848796 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ecc0b-b481-4776-aae4-a7575fc5ab60 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:00.853428 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:00.854044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.854331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.854614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.855331 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:11:00.855643 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:00.857288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:00.869601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1425/2828] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:11:00.873853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903209a-30ab-4e7b-8f42-2ac7f958d9f5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:00.883085 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903209a-30ab-4e7b-8f42-2ac7f958d9f5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:00.887630 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:00.888230 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.888510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.888825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.890243 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:00.892118 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:00.918788 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1404/2829] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:00 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:00.923004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:00.932907 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7f03a-9a5b-43ca-8c37-014b5f49e773 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:00.989776 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:00.990394 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:00.990676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:00.990953 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:00.991602 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:00.993638 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.015893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1426/2830] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:00 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:01.019903 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.031188 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967b35b-204c-404a-8acb-96674c319085 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.035711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.036314 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.036698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.036977 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.037657 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.039227 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.049052 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1405/2831] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:01.052822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.062722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d77c7-f797-4bfc-99bb-c1c224e5f52b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.067532 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd8680-8466-487a-a46a-c7e789e7e274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.068132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd8680-8466-487a-a46a-c7e789e7e274 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.068413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd8680-8466-487a-a46a-c7e789e7e274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.068725 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd8680-8466-487a-a46a-c7e789e7e274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.069406 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd8680-8466-487a-a46a-c7e789e7e274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.070971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd8680-8466-487a-a46a-c7e789e7e274 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.082629 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1427/2832] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:01.086896 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.098265 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce89b82-e955-4826-ad84-1acf6318ab69 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.102932 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.103587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.103872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.104151 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] PATH_INFO: `/v3/projects/7ad5d791dbf8499bbf7cc88cadec2d21/users/0d2e0cf1f1024d98a5fe1a53aa3db53c/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.118604 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad5d791dbf8499bbf7cc88cadec2d21, user_id=0d2e0cf1f1024d98a5fe1a53aa3db53c, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.121296 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.134527 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1406/2833] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:01 2022] PUT /identity/v3/projects/7ad5d791dbf8499bbf7cc88cadec2d21/users/0d2e0cf1f1024d98a5fe1a53aa3db53c/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:11:01.145360 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cea844-a78a-458d-ab95-132f197cdc89 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.145701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cea844-a78a-458d-ab95-132f197cdc89 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.146031 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cea844-a78a-458d-ab95-132f197cdc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.173657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-25cea844-a78a-458d-ab95-132f197cdc89 None None] MFA Rules not processed for user `0d2e0cf1f1024d98a5fe1a53aa3db53c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:01.174880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-25cea844-a78a-458d-ab95-132f197cdc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:01.201230 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:01.241004 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1428/2834] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:01.245576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.267234 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:01.309129 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ff0e6-c87b-4920-8839-cd18dc73e9cd admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.314633 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.315244 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.315528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.315805 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.316466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.318391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.345210 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1407/2835] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:01 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:01.349311 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cea844-a78a-458d-ab95-132f197cdc89 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.360956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cea844-a78a-458d-ab95-132f197cdc89 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.417865 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.418442 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.418726 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.419032 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.419687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.421714 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.442797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1429/2836] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:01 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:01.446848 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.457601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17f8ff-bddc-43a1-bc2c-d9b338417feb admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.462161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.463020 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.463303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.463576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.464253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.465893 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.476186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1408/2837] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:11:01.480215 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.490918 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a24238-caad-45ee-99ee-56dae18dd0b0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.495455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.496100 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.496383 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.496699 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] PATH_INFO: `/v3/projects/b085df15c9174c9d80d01bc5df69a8a9/users/c6c32b83e5894d23ad9177e7a1388cd5/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.512817 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b085df15c9174c9d80d01bc5df69a8a9, user_id=c6c32b83e5894d23ad9177e7a1388cd5, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.515645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.528151 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1430/2838] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:01 2022] PUT /identity/v3/projects/b085df15c9174c9d80d01bc5df69a8a9/users/c6c32b83e5894d23ad9177e7a1388cd5/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:01.532368 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.542440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d593e-ffb9-44a3-b634-923ecef81162 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.599267 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870698d-ae62-40ec-a182-8f15890fbbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.599885 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870698d-ae62-40ec-a182-8f15890fbbde admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.600166 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870698d-ae62-40ec-a182-8f15890fbbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.600439 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870698d-ae62-40ec-a182-8f15890fbbde admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.601159 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870698d-ae62-40ec-a182-8f15890fbbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.602763 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870698d-ae62-40ec-a182-8f15890fbbde admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.612946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1409/2839] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:01.617161 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.628142 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f7e7c-a6c8-4995-b1c6-b134daf972ad admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.632744 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.633407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.633687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.633962 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] PATH_INFO: `/v3/projects/b085df15c9174c9d80d01bc5df69a8a9/users/c6c32b83e5894d23ad9177e7a1388cd5/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.635905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b085df15c9174c9d80d01bc5df69a8a9, user_id=c6c32b83e5894d23ad9177e7a1388cd5, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.638548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.648527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1431/2840] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:01 2022] PUT /identity/v3/projects/b085df15c9174c9d80d01bc5df69a8a9/users/c6c32b83e5894d23ad9177e7a1388cd5/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:01.658716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c059c69-1de4-47a7-b447-8e016b8341be None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.659197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c059c69-1de4-47a7-b447-8e016b8341be None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.659315 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c059c69-1de4-47a7-b447-8e016b8341be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.688024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-7c059c69-1de4-47a7-b447-8e016b8341be None None] MFA Rules not processed for user `c6c32b83e5894d23ad9177e7a1388cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:01.689233 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-7c059c69-1de4-47a7-b447-8e016b8341be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:01.758795 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1410/2841] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:01.769858 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:01.771667 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:01.828678 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecfc76-1ce7-4524-8fea-b895e3bcc7da admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:01.832296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce79ba3-5202-4ae7-b4c7-a72f47a82fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:01.833011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce79ba3-5202-4ae7-b4c7-a72f47a82fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:01.833295 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce79ba3-5202-4ae7-b4c7-a72f47a82fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:01.833574 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce79ba3-5202-4ae7-b4c7-a72f47a82fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:01.843380 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce79ba3-5202-4ae7-b4c7-a72f47a82fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:01.844843 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce79ba3-5202-4ae7-b4c7-a72f47a82fc6 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:01.854134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1432/2842] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:03.195755 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c059c69-1de4-47a7-b447-8e016b8341be None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:03.213148 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:03.261511 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c059c69-1de4-47a7-b447-8e016b8341be None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:03.266319 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:03.266961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:03.267245 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:03.267524 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] PATH_INFO: `/v3/users/0d2e0cf1f1024d98a5fe1a53aa3db53c` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:03.268711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2e0cf1f1024d98a5fe1a53aa3db53c)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:03.270605 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:03.315149 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2e0cf1f1024d98a5fe1a53aa3db53c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.315780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2e0cf1f1024d98a5fe1a53aa3db53c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.327875 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2e0cf1f1024d98a5fe1a53aa3db53c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.334803 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2e0cf1f1024d98a5fe1a53aa3db53c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.335443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2e0cf1f1024d98a5fe1a53aa3db53c'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.348664 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1411/2843] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:03 2022] DELETE /identity/v3/users/0d2e0cf1f1024d98a5fe1a53aa3db53c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:03.353308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:03.353770 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:03.353869 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:03.372902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:03.374455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:03.390956 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:03.433822 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1433/2844] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:03.443443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:03.444897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:03.503510 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c037-b0ab-46bb-9bfc-91cf903b054f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:03.507116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:03.507730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:03.508011 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:03.508287 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:03.517642 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:03.519058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:03.528405 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1412/2845] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:03.791695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:03.793237 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:03.808113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654bfe-8ec6-4653-86d9-5fd5510d8e9a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:03.813547 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:03.814197 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:03.814477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:03.814740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] PATH_INFO: `/v3/projects/7ad5d791dbf8499bbf7cc88cadec2d21` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:03.816002 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad5d791dbf8499bbf7cc88cadec2d21)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:03.817929 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:03.865249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad5d791dbf8499bbf7cc88cadec2d21', 'request_id': 'req-9ce8d216-6b52-4c9a-8585-923a9140da7f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.872591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad5d791dbf8499bbf7cc88cadec2d21', 'request_id': 'req-9ce8d216-6b52-4c9a-8585-923a9140da7f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.879035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad5d791dbf8499bbf7cc88cadec2d21', 'request_id': 'req-9ce8d216-6b52-4c9a-8585-923a9140da7f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.890117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] The token cache is being invalidate because project 7ad5d791dbf8499bbf7cc88cadec2d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:03.890411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.891355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:03.892905 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1434/2846] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:03 2022] DELETE /identity/v3/projects/7ad5d791dbf8499bbf7cc88cadec2d21 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:03.896950 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:03.907462 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3f9d3-009b-420a-9275-80d50f8bff69 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:03.939544 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:03.964513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:03.965190 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:03.965477 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:03.965758 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] PATH_INFO: `/v3/users/c6c32b83e5894d23ad9177e7a1388cd5` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:03.966924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c32b83e5894d23ad9177e7a1388cd5)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:03.969598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.014609 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c32b83e5894d23ad9177e7a1388cd5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.015316 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c32b83e5894d23ad9177e7a1388cd5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.027125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c32b83e5894d23ad9177e7a1388cd5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.035019 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c32b83e5894d23ad9177e7a1388cd5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.035613 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-4a1d9666-79b2-48ee-a8a8-351b3a8f5d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c32b83e5894d23ad9177e7a1388cd5'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.048376 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1413/2847] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:03 2022] DELETE /identity/v3/users/c6c32b83e5894d23ad9177e7a1388cd5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:04.318146 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:04.319686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:04.333474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8d216-6b52-4c9a-8585-923a9140da7f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:04.392011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:04.392729 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.392971 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.393251 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] PATH_INFO: `/v3/projects/b085df15c9174c9d80d01bc5df69a8a9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.394421 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b085df15c9174c9d80d01bc5df69a8a9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:04.396325 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.441385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:04.443012 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b085df15c9174c9d80d01bc5df69a8a9', 'request_id': 'req-ae4509e5-a442-4036-98f9-530eef56aad8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.450336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b085df15c9174c9d80d01bc5df69a8a9', 'request_id': 'req-ae4509e5-a442-4036-98f9-530eef56aad8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.457705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b085df15c9174c9d80d01bc5df69a8a9', 'request_id': 'req-ae4509e5-a442-4036-98f9-530eef56aad8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.468888 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] The token cache is being invalidate because project b085df15c9174c9d80d01bc5df69a8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:04.469179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.469810 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ae4509e5-a442-4036-98f9-530eef56aad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:04.471296 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1435/2848] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:04 2022] DELETE /identity/v3/projects/b085df15c9174c9d80d01bc5df69a8a9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:04.476293 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebf3c3-5255-4e8e-9452-07b7a977b25e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.476775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebf3c3-5255-4e8e-9452-07b7a977b25e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.476930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebf3c3-5255-4e8e-9452-07b7a977b25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.497538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-52ebf3c3-5255-4e8e-9452-07b7a977b25e None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:04.499360 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-52ebf3c3-5255-4e8e-9452-07b7a977b25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:04.559039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1414/2849] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:04.567580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f7326-3919-4171-81b1-2b79085d4ee1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.567814 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f7326-3919-4171-81b1-2b79085d4ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.568101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f7326-3919-4171-81b1-2b79085d4ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.588501 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c89f7326-3919-4171-81b1-2b79085d4ee1 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:04.589734 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c89f7326-3919-4171-81b1-2b79085d4ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:04.598724 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1436/2850] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:04.602985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebf3c3-5255-4e8e-9452-07b7a977b25e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:04.612867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebf3c3-5255-4e8e-9452-07b7a977b25e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:04.617407 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:04.618013 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.618295 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.618572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.619252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:11:04.619554 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:04.621175 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.632859 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1415/2851] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:11:04.637127 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f7326-3919-4171-81b1-2b79085d4ee1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:04.647266 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f7326-3919-4171-81b1-2b79085d4ee1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:04.651906 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:04.652511 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.652821 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.653097 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.653747 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:04.656592 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.683586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1437/2852] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:04 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:04.688122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:04.698185 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b844bb-5aa0-4a40-b16f-b3f27ad7f870 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:04.742440 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:04.758889 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:04.759513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.759793 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.760065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.760744 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:04.762629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.784419 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1416/2853] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:04 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:04.788767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:04.799199 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507ed09-3080-4b68-9f37-01bd8225e4fd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:04.803836 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:04.804448 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.804767 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.805047 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.805736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:04.807318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.817103 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1438/2854] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:04.820958 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:04.831400 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cd58f-8f37-47f3-bfdb-60fd1e1c5e63 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:04.836000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d816d-eb14-40ec-af60-6c1238e075c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:04.836647 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d816d-eb14-40ec-af60-6c1238e075c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.836935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d816d-eb14-40ec-af60-6c1238e075c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.837214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d816d-eb14-40ec-af60-6c1238e075c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.838161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d816d-eb14-40ec-af60-6c1238e075c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:04.839742 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d816d-eb14-40ec-af60-6c1238e075c6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.851066 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1417/2855] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:04.855061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:04.865354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baa344-954a-4cf3-8a7e-525b3abc8743 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:04.870005 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:04.870658 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.870946 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.871227 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] PATH_INFO: `/v3/projects/a1cfec793e0846f28f71a0771bec7889/users/2ac5912d1a114bccbad2274b55c77459/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.886628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cfec793e0846f28f71a0771bec7889, user_id=2ac5912d1a114bccbad2274b55c77459, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:04.890169 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:04.903873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1439/2856] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:04 2022] PUT /identity/v3/projects/a1cfec793e0846f28f71a0771bec7889/users/2ac5912d1a114bccbad2274b55c77459/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:04.914022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be378e41-2577-46cf-89a4-407ca820a82a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:04.914341 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be378e41-2577-46cf-89a4-407ca820a82a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:04.914619 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be378e41-2577-46cf-89a4-407ca820a82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:04.942383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-be378e41-2577-46cf-89a4-407ca820a82a None None] MFA Rules not processed for user `2ac5912d1a114bccbad2274b55c77459`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:04.943587 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-be378e41-2577-46cf-89a4-407ca820a82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:05.007253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1418/2857] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:05.011693 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.073051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe76a-f60c-4309-a734-93c6afeb63e3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.078260 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.078877 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.079160 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.079438 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.080099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.082028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.108054 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1440/2858] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:05 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:05.112200 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be378e41-2577-46cf-89a4-407ca820a82a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.122904 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be378e41-2577-46cf-89a4-407ca820a82a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.178569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.179187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.179474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.179756 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.180421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.182391 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.203370 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1419/2859] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:05 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:05.207515 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.218838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da040411-deec-4f2a-bd82-1bfee45f3cb5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.223583 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.224194 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.224478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.224789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.225475 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.227064 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.238968 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1441/2860] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:05.242987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.253641 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb5c9b-bdb1-48fc-911e-246a62e50648 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.258335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.258993 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.259282 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.259567 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] PATH_INFO: `/v3/projects/c7c987fdf29d4b758ceea1025dc7ddb8/users/b5b49ecc56bb4520b35af28e2edb11cf/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.275212 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c987fdf29d4b758ceea1025dc7ddb8, user_id=b5b49ecc56bb4520b35af28e2edb11cf, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.277912 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.290820 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1420/2861] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:05 2022] PUT /identity/v3/projects/c7c987fdf29d4b758ceea1025dc7ddb8/users/b5b49ecc56bb4520b35af28e2edb11cf/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:11:05.294805 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.305910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb779b1c-2e43-4f44-91c7-03c800b70bb4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.350355 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:05.365180 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5744b-a128-4f08-81f6-1805e5b6169e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.365797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5744b-a128-4f08-81f6-1805e5b6169e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.366081 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5744b-a128-4f08-81f6-1805e5b6169e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.366359 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5744b-a128-4f08-81f6-1805e5b6169e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.367040 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5744b-a128-4f08-81f6-1805e5b6169e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.368686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5744b-a128-4f08-81f6-1805e5b6169e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.378562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1442/2862] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:05.382736 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.393607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cea108-5378-4f96-a1fa-869ef606a8e6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.398252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.398908 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.399195 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.399474 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] PATH_INFO: `/v3/projects/c7c987fdf29d4b758ceea1025dc7ddb8/users/b5b49ecc56bb4520b35af28e2edb11cf/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.401482 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c987fdf29d4b758ceea1025dc7ddb8, user_id=b5b49ecc56bb4520b35af28e2edb11cf, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.404101 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.414918 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1421/2863] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:05 2022] PUT /identity/v3/projects/c7c987fdf29d4b758ceea1025dc7ddb8/users/b5b49ecc56bb4520b35af28e2edb11cf/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:05.426021 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6021268-f4bf-4d5b-92b2-af095e68dfd0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.426478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6021268-f4bf-4d5b-92b2-af095e68dfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.426563 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6021268-f4bf-4d5b-92b2-af095e68dfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.454609 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e6021268-f4bf-4d5b-92b2-af095e68dfd0 None None] MFA Rules not processed for user `b5b49ecc56bb4520b35af28e2edb11cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:05.455820 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e6021268-f4bf-4d5b-92b2-af095e68dfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:05.523608 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1443/2864] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:05.535558 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.537336 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:05.543808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:05.596569 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a1c0f-4798-4700-bc29-a807b12f01f6 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.600199 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32bb87-a16b-43be-9660-0d85e5845683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.600897 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32bb87-a16b-43be-9660-0d85e5845683 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.601125 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32bb87-a16b-43be-9660-0d85e5845683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.601396 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32bb87-a16b-43be-9660-0d85e5845683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.610383 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32bb87-a16b-43be-9660-0d85e5845683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.611794 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32bb87-a16b-43be-9660-0d85e5845683 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.621107 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1422/2865] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:05.687557 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6021268-f4bf-4d5b-92b2-af095e68dfd0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.751676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6021268-f4bf-4d5b-92b2-af095e68dfd0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:05.756243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:05.756919 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.757206 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.757484 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] PATH_INFO: `/v3/users/2ac5912d1a114bccbad2274b55c77459` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.758661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac5912d1a114bccbad2274b55c77459)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:05.760606 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:05.807387 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac5912d1a114bccbad2274b55c77459'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:05.814752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac5912d1a114bccbad2274b55c77459'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:05.824698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:05.827367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac5912d1a114bccbad2274b55c77459'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:05.835173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac5912d1a114bccbad2274b55c77459'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:05.835815 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac5912d1a114bccbad2274b55c77459'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:05.842480 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1444/2866] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:05 2022] DELETE /identity/v3/users/2ac5912d1a114bccbad2274b55c77459 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:05.847039 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:05.847331 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:05.847639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:05.866872 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:05.868000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:05.925869 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1423/2867] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:05.935652 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:05.937245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:06.000378 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f682ff-5068-4e74-b1dd-1bc9e8b162af admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:06.003934 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:06.004560 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:06.004860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:06.005134 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:06.014431 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:06.015935 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:06.025178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1445/2868] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:06.294856 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:06.296878 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:06.308798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884deff7-250c-4b12-bc78-cc8f6d91905f None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:06.314704 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:06.315357 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:06.315643 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:06.315922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] PATH_INFO: `/v3/projects/a1cfec793e0846f28f71a0771bec7889` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:06.317117 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cfec793e0846f28f71a0771bec7889)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:06.319017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:06.366189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cfec793e0846f28f71a0771bec7889', 'request_id': 'req-a7776a15-788a-4701-a2a0-ecfc805c3ad2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.372413 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cfec793e0846f28f71a0771bec7889', 'request_id': 'req-a7776a15-788a-4701-a2a0-ecfc805c3ad2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.380844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cfec793e0846f28f71a0771bec7889', 'request_id': 'req-a7776a15-788a-4701-a2a0-ecfc805c3ad2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.391468 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] The token cache is being invalidate because project a1cfec793e0846f28f71a0771bec7889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:06.391751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.392366 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.393891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1424/2869] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:06 2022] DELETE /identity/v3/projects/a1cfec793e0846f28f71a0771bec7889 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:06.398117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:06.408436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3f85f-907d-40fa-8cb1-ac0882effbad service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:06.463864 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:06.464485 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:06.464803 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:06.465084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] PATH_INFO: `/v3/users/b5b49ecc56bb4520b35af28e2edb11cf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:06.466242 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b49ecc56bb4520b35af28e2edb11cf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:06.468853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:06.514301 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b49ecc56bb4520b35af28e2edb11cf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.521144 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b49ecc56bb4520b35af28e2edb11cf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.533477 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b49ecc56bb4520b35af28e2edb11cf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.540797 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b49ecc56bb4520b35af28e2edb11cf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.541407 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c9cc6a47-1e9f-44be-8fb1-5948841db914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b49ecc56bb4520b35af28e2edb11cf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.548262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1446/2870] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:06 2022] DELETE /identity/v3/users/b5b49ecc56bb4520b35af28e2edb11cf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:06.817880 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:06.819334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:06.833115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7776a15-788a-4701-a2a0-ecfc805c3ad2 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:06.864183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:06.893590 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:06.894139 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:06.894422 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:06.894697 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] PATH_INFO: `/v3/projects/c7c987fdf29d4b758ceea1025dc7ddb8` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:06.895845 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c987fdf29d4b758ceea1025dc7ddb8)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:06.897776 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:06.945003 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c987fdf29d4b758ceea1025dc7ddb8', 'request_id': 'req-14f9a057-02d0-4d84-81fc-1105b890585a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.951027 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c987fdf29d4b758ceea1025dc7ddb8', 'request_id': 'req-14f9a057-02d0-4d84-81fc-1105b890585a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.959161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c987fdf29d4b758ceea1025dc7ddb8', 'request_id': 'req-14f9a057-02d0-4d84-81fc-1105b890585a'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.970252 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] The token cache is being invalidate because project c7c987fdf29d4b758ceea1025dc7ddb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:06.970540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.971194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-14f9a057-02d0-4d84-81fc-1105b890585a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:06.972716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1425/2871] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:06 2022] DELETE /identity/v3/projects/c7c987fdf29d4b758ceea1025dc7ddb8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:06.977601 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa355014-cb17-40c2-9212-24aa29275f1c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:06.977893 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa355014-cb17-40c2-9212-24aa29275f1c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:06.978178 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa355014-cb17-40c2-9212-24aa29275f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:06.997701 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fa355014-cb17-40c2-9212-24aa29275f1c None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:06.999114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fa355014-cb17-40c2-9212-24aa29275f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:07.004059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:07.062939 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1447/2872] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:07.070765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1071fd-3a89-4d66-bd6f-aea3ff9191ba None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.071053 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1071fd-3a89-4d66-bd6f-aea3ff9191ba None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.071334 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1071fd-3a89-4d66-bd6f-aea3ff9191ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.095621 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-cd1071fd-3a89-4d66-bd6f-aea3ff9191ba None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:07.096868 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-cd1071fd-3a89-4d66-bd6f-aea3ff9191ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:07.104751 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1426/2873] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:07.109824 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa355014-cb17-40c2-9212-24aa29275f1c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.119736 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa355014-cb17-40c2-9212-24aa29275f1c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.124408 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.125056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.125336 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.125610 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.126287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:11:07.126590 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.128250 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.140084 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1448/2874] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:11:07.144327 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1071fd-3a89-4d66-bd6f-aea3ff9191ba None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.153285 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1071fd-3a89-4d66-bd6f-aea3ff9191ba None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.158593 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.159189 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.159467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.159741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.160394 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.162304 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.190377 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1427/2875] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:07.194470 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.205148 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d139acc-ad9d-4922-8958-39419f44a8f9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.260616 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.261223 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.261504 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.261781 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.262443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.264419 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.285210 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1449/2876] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:07.289325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.301520 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38ac69-9e4d-40bc-9c90-906bb461ca31 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.306253 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.306860 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.307141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.307417 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.308094 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.309698 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.319479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1428/2877] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:07.323350 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.333443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6402c-e916-4a9e-b02b-8d29ca5bd995 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.338249 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424eeb9-6abd-422a-8c7b-a2c01fabe865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.338847 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eeb9-6abd-422a-8c7b-a2c01fabe865 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.339130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eeb9-6abd-422a-8c7b-a2c01fabe865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.339405 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eeb9-6abd-422a-8c7b-a2c01fabe865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.340079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424eeb9-6abd-422a-8c7b-a2c01fabe865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.341688 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424eeb9-6abd-422a-8c7b-a2c01fabe865 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.353873 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1450/2878] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:07.358628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.368653 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b18b02-85bb-48ef-bdca-bc9557016441 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.373303 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.373961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.374246 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.374523 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] PATH_INFO: `/v3/projects/3a4f722d2fed4143809cf40d665e05d1/users/494e092287e14e269b4af85918422081/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.388808 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4f722d2fed4143809cf40d665e05d1, user_id=494e092287e14e269b4af85918422081, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.391443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.403914 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1429/2879] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:07 2022] PUT /identity/v3/projects/3a4f722d2fed4143809cf40d665e05d1/users/494e092287e14e269b4af85918422081/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:07.414697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ef14-5daf-45b1-a1ae-c2f22b085e31 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.414982 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ef14-5daf-45b1-a1ae-c2f22b085e31 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.415258 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ef14-5daf-45b1-a1ae-c2f22b085e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.442874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-06f9ef14-5daf-45b1-a1ae-c2f22b085e31 None None] MFA Rules not processed for user `494e092287e14e269b4af85918422081`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:07.444052 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-06f9ef14-5daf-45b1-a1ae-c2f22b085e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:07.507051 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1451/2880] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:07.511588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.570286 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:07.574194 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda7dc-56aa-47de-98f2-114c48d17253 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.580815 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.581616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.582081 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.582427 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.583250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.585225 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.612054 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1430/2881] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:07.616138 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9ef14-5daf-45b1-a1ae-c2f22b085e31 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.622800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:07.628614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9ef14-5daf-45b1-a1ae-c2f22b085e31 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.684682 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.685225 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.685508 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.685811 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.686478 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.688367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.710460 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1452/2882] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:11:07.714764 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.725819 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a109d19-00b1-473f-a8ad-2016e2d28d8a admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.730470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.731110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.731389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.731662 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.732466 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.734088 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.745677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1431/2883] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:07.750035 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.759999 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0edcbe-1e74-4096-9857-81045afe497c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.765262 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24699785-772c-4bf8-9cca-42eb405a7530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.765848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24699785-772c-4bf8-9cca-42eb405a7530 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.766129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24699785-772c-4bf8-9cca-42eb405a7530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.766402 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24699785-772c-4bf8-9cca-42eb405a7530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.767074 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24699785-772c-4bf8-9cca-42eb405a7530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.768668 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24699785-772c-4bf8-9cca-42eb405a7530 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.778319 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1453/2884] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:07.783078 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:07.792933 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050d69c-f932-4816-9c61-52fd3568ea2f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:07.798179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:07.798825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.799115 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.799389 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] PATH_INFO: `/v3/projects/b1b60bef15bd4391b7069e42304084fe/users/22f44b3d34464eb28ce0e6cddcbe5851/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.814158 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b60bef15bd4391b7069e42304084fe, user_id=22f44b3d34464eb28ce0e6cddcbe5851, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:07.816811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:07.829361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1432/2885] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:07 2022] PUT /identity/v3/projects/b1b60bef15bd4391b7069e42304084fe/users/22f44b3d34464eb28ce0e6cddcbe5851/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 07:11:07.840789 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c2ccb-bc0e-4e12-ab1a-b506c2e13743 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:07.841079 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c2ccb-bc0e-4e12-ab1a-b506c2e13743 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:07.841358 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c2ccb-bc0e-4e12-ab1a-b506c2e13743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:07.869139 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7f3c2ccb-bc0e-4e12-ab1a-b506c2e13743 None None] MFA Rules not processed for user `22f44b3d34464eb28ce0e6cddcbe5851`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:07.870332 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7f3c2ccb-bc0e-4e12-ab1a-b506c2e13743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:07.934623 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1454/2886] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:07.939798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.001675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e44314-63a7-42ca-94f1-f3dce9dc7bcf admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.006442 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.007061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.007348 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.007628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.008301 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.010269 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.037463 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1433/2887] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:07 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:08.041550 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c2ccb-bc0e-4e12-ab1a-b506c2e13743 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.053173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c2ccb-bc0e-4e12-ab1a-b506c2e13743 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.110750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.111363 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.111646 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.111920 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.112599 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.114491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.135291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1455/2888] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:08 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:08.139549 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.149931 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c178-8eb0-4ce5-acc3-096515b144b4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.155038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.155651 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.155935 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.156211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.156927 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.158504 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.168891 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1434/2889] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:11:08.173132 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.178261 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:08.184519 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f86012-e521-4b2c-bff1-357d02bbdb46 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.190465 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.191114 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.191396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.191676 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] PATH_INFO: `/v3/projects/fcaa1379f51e4a62b055e6d102dca68b/users/9ad2b7dd9a374d51be7bb418a095253b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.207474 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcaa1379f51e4a62b055e6d102dca68b, user_id=9ad2b7dd9a374d51be7bb418a095253b, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.210195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.223759 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1456/2890] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:08 2022] PUT /identity/v3/projects/fcaa1379f51e4a62b055e6d102dca68b/users/9ad2b7dd9a374d51be7bb418a095253b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:08.227707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.239317 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f154d78-4cf0-4632-86ff-31b1bee54f23 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.296441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11495e-63d6-4f62-9304-eae7730f3102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.297083 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11495e-63d6-4f62-9304-eae7730f3102 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.297361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11495e-63d6-4f62-9304-eae7730f3102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.297658 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11495e-63d6-4f62-9304-eae7730f3102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.298335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11495e-63d6-4f62-9304-eae7730f3102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.299934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11495e-63d6-4f62-9304-eae7730f3102 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.309831 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1435/2891] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:08.314617 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.324868 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353bee-0b71-407b-a07d-240d2a02f843 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.329553 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.330213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.330499 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.330782 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] PATH_INFO: `/v3/projects/fcaa1379f51e4a62b055e6d102dca68b/users/9ad2b7dd9a374d51be7bb418a095253b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.332834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcaa1379f51e4a62b055e6d102dca68b, user_id=9ad2b7dd9a374d51be7bb418a095253b, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.335445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.346011 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1457/2892] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:08 2022] PUT /identity/v3/projects/fcaa1379f51e4a62b055e6d102dca68b/users/9ad2b7dd9a374d51be7bb418a095253b/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:08.356191 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.356486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.356807 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.386258 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] MFA Rules not processed for user `9ad2b7dd9a374d51be7bb418a095253b`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:08.387716 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:08.392894 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:08.455277 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1436/2893] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:08.466687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.468718 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:08.529252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2497e-6b0e-4960-a14d-2998099b0be5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.533089 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.533710 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.533996 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.534280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.543651 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.545707 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.555287 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1458/2894] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:08.597783 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:08.599196 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:08.611410 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f22792-567f-4542-8f84-3917f9cf927e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:08.615104 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aadbe-bb6a-412d-8c3d-1cd4285484b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:08.615730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aadbe-bb6a-412d-8c3d-1cd4285484b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:08.616016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aadbe-bb6a-412d-8c3d-1cd4285484b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:08.616298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aadbe-bb6a-412d-8c3d-1cd4285484b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:08.673836 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aadbe-bb6a-412d-8c3d-1cd4285484b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:08.675257 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aadbe-bb6a-412d-8c3d-1cd4285484b8 service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:08.684122 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1437/2895] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:11.552988 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:11.615039 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0fffb-cb13-4a53-aba5-e9922874e986 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:11.619483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:11.620113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:11.620393 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:11.620708 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] PATH_INFO: `/v3/users/494e092287e14e269b4af85918422081` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:11.621880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=494e092287e14e269b4af85918422081)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:11.623860 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:11.668491 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '494e092287e14e269b4af85918422081'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:11.676198 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '494e092287e14e269b4af85918422081'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:11.686130 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:11.688257 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '494e092287e14e269b4af85918422081'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:11.695614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '494e092287e14e269b4af85918422081'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:11.696562 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '494e092287e14e269b4af85918422081'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:11.703436 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1459/2896] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:11 2022] DELETE /identity/v3/users/494e092287e14e269b4af85918422081 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:11.707987 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:11.708441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:11.708674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:11.728719 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:11.729862 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:11.789278 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1438/2897] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:11:11.799570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:11.801023 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:11.859627 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177722d-aa83-4047-9032-119ae2b0cc32 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:11.863244 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:11.863853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:11.864131 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:11.864404 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:11.873624 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:11.875059 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:11.884483 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1460/2898] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:11:12.142838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:12.144397 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:12.158739 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b2d0-9508-46f6-be25-2b11be03f950 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:12.163467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:12.164120 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:12.164437 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:12.164777 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] PATH_INFO: `/v3/projects/3a4f722d2fed4143809cf40d665e05d1` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:12.165946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4f722d2fed4143809cf40d665e05d1)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:12.168033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:12.205961 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:12.216087 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4f722d2fed4143809cf40d665e05d1', 'request_id': 'req-3e064e1f-16d8-4142-92d5-ffa9146f0be2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.223014 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4f722d2fed4143809cf40d665e05d1', 'request_id': 'req-3e064e1f-16d8-4142-92d5-ffa9146f0be2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.229550 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4f722d2fed4143809cf40d665e05d1', 'request_id': 'req-3e064e1f-16d8-4142-92d5-ffa9146f0be2'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.239946 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] The token cache is being invalidate because project 3a4f722d2fed4143809cf40d665e05d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:12.240236 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.240881 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.242415 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1439/2899] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:12 2022] DELETE /identity/v3/projects/3a4f722d2fed4143809cf40d665e05d1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:12.246410 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:12.256069 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb9c6d-9c5d-4dc9-af6d-a206c76c1291 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:12.311292 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:12.311827 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:12.312109 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:12.312385 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] PATH_INFO: `/v3/users/22f44b3d34464eb28ce0e6cddcbe5851` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:12.313570 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f44b3d34464eb28ce0e6cddcbe5851)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:12.315472 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:12.361126 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f44b3d34464eb28ce0e6cddcbe5851'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.367938 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f44b3d34464eb28ce0e6cddcbe5851'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.380917 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f44b3d34464eb28ce0e6cddcbe5851'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.388046 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f44b3d34464eb28ce0e6cddcbe5851'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.388696 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f44b3d34464eb28ce0e6cddcbe5851'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.395307 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1461/2900] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:12 2022] DELETE /identity/v3/users/22f44b3d34464eb28ce0e6cddcbe5851 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:12.664575 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:12.665992 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:12.679816 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e064e1f-16d8-4142-92d5-ffa9146f0be2 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:12.746361 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:12.747017 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:12.747297 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:12.747571 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] PATH_INFO: `/v3/projects/b1b60bef15bd4391b7069e42304084fe` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:12.748825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b60bef15bd4391b7069e42304084fe)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:12.750713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:12.798143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b60bef15bd4391b7069e42304084fe', 'request_id': 'req-78a112bd-df89-4647-8e36-1b40b7dc8073'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.804110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b60bef15bd4391b7069e42304084fe', 'request_id': 'req-78a112bd-df89-4647-8e36-1b40b7dc8073'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.811224 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b60bef15bd4391b7069e42304084fe', 'request_id': 'req-78a112bd-df89-4647-8e36-1b40b7dc8073'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.821429 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] The token cache is being invalidate because project b1b60bef15bd4391b7069e42304084fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:12.821722 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.822352 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.823847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1440/2901] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:12 2022] DELETE /identity/v3/projects/b1b60bef15bd4391b7069e42304084fe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:12.828481 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:12.829941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:12.843660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439d89d-42d9-4efd-bad4-7e3cb05ade39 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:12.848203 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:12.902061 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:12.902705 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:12.902994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:12.903277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] PATH_INFO: `/v3/users/9ad2b7dd9a374d51be7bb418a095253b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:12.904451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad2b7dd9a374d51be7bb418a095253b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:12.907185 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:12.952521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad2b7dd9a374d51be7bb418a095253b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.959569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad2b7dd9a374d51be7bb418a095253b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.972154 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad2b7dd9a374d51be7bb418a095253b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.979429 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad2b7dd9a374d51be7bb418a095253b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.980058 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a3c3427-ab3b-48bd-ac81-3142c0834330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad2b7dd9a374d51be7bb418a095253b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:12.986063 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1462/2902] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:12 2022] DELETE /identity/v3/users/9ad2b7dd9a374d51be7bb418a095253b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:13.254000 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:13.255432 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:13.270061 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a112bd-df89-4647-8e36-1b40b7dc8073 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:13.321170 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:13.329727 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:13.330576 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.331024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.331517 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] PATH_INFO: `/v3/projects/fcaa1379f51e4a62b055e6d102dca68b` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.332726 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcaa1379f51e4a62b055e6d102dca68b)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:13.334634 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:13.380283 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcaa1379f51e4a62b055e6d102dca68b', 'request_id': 'req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:13.386157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcaa1379f51e4a62b055e6d102dca68b', 'request_id': 'req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:13.393570 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcaa1379f51e4a62b055e6d102dca68b', 'request_id': 'req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:13.403930 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] The token cache is being invalidate because project fcaa1379f51e4a62b055e6d102dca68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80162) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:13.404214 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:13.404847 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-c332363d-2ba3-41f6-bfe0-b44ea3c78b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:13.406501 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1441/2903] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:13 2022] DELETE /identity/v3/projects/fcaa1379f51e4a62b055e6d102dca68b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:13.412409 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6eaa1-2d3b-4a4e-9018-3db559bd519b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.412752 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6eaa1-2d3b-4a4e-9018-3db559bd519b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.413030 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6eaa1-2d3b-4a4e-9018-3db559bd519b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.432300 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f3e6eaa1-2d3b-4a4e-9018-3db559bd519b None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:13.433548 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f3e6eaa1-2d3b-4a4e-9018-3db559bd519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:13.492850 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1463/2904] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:13.501852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa102-f71e-4f3a-9cbd-e724c459d343 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.502144 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa102-f71e-4f3a-9cbd-e724c459d343 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.502426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa102-f71e-4f3a-9cbd-e724c459d343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.522179 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5f9aa102-f71e-4f3a-9cbd-e724c459d343 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:13.523364 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5f9aa102-f71e-4f3a-9cbd-e724c459d343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:13.531299 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1442/2905] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:13.535502 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6eaa1-2d3b-4a4e-9018-3db559bd519b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:13.545902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6eaa1-2d3b-4a4e-9018-3db559bd519b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:13.550493 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:13.551099 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.551384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.551661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.552346 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:11:13.552695 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:13.554280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:13.566559 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1464/2906] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 07:11:13.570854 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa102-f71e-4f3a-9cbd-e724c459d343 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:13.580291 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa102-f71e-4f3a-9cbd-e724c459d343 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:13.585028 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:13.585637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.585924 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.586205 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.586876 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:13.589089 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:13.618513 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1443/2907] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:13 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:11:13.622494 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:13.633389 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b93624-2a99-4574-8b4e-826631a7ef60 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:13.644703 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:13.690800 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:13.691411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.691698 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.691967 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.692640 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:13.694538 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:13.716551 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1465/2908] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:13 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:13.720780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:13.731380 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db1b5-1d01-4be0-9b44-8da8ea2941d4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:13.736853 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:13.737470 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.737782 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.738058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.738741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:13.740321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:13.751143 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1444/2909] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:13.755102 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:13.765192 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58b1f-6492-4ae7-bcbe-45c12382a9f6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:13.769987 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84ea3c-be10-4c1a-aec4-516e9cbecc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:13.770622 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ea3c-be10-4c1a-aec4-516e9cbecc72 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.770910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ea3c-be10-4c1a-aec4-516e9cbecc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.771186 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ea3c-be10-4c1a-aec4-516e9cbecc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.771891 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84ea3c-be10-4c1a-aec4-516e9cbecc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:13.773527 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84ea3c-be10-4c1a-aec4-516e9cbecc72 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:13.784910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1466/2910] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 07:11:13.789024 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:13.799424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96090a1c-462a-4d88-b089-76b13bf5a51e admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:13.804887 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:13.805541 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.805827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.806102 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] PATH_INFO: `/v3/projects/68dd61cf13794ccc91c3c86c686b79f7/users/9b9429309b2b4ae482798bfb6f21a193/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.821041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68dd61cf13794ccc91c3c86c686b79f7, user_id=9b9429309b2b4ae482798bfb6f21a193, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:13.823663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:13.836161 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1445/2911] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:13 2022] PUT /identity/v3/projects/68dd61cf13794ccc91c3c86c686b79f7/users/9b9429309b2b4ae482798bfb6f21a193/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:13.849344 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d973255-b2a1-49e8-aa68-5cf56b806fa5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:13.849806 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d973255-b2a1-49e8-aa68-5cf56b806fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:13.849889 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d973255-b2a1-49e8-aa68-5cf56b806fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:13.878449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3d973255-b2a1-49e8-aa68-5cf56b806fa5 None None] MFA Rules not processed for user `9b9429309b2b4ae482798bfb6f21a193`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:13.879687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d973255-b2a1-49e8-aa68-5cf56b806fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:13.944252 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1467/2912] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:13.948967 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.012484 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25f94d-3864-4815-98b6-2cfbaedf69e0 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.017699 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.018318 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.018602 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.018879 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.019535 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.021467 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.049628 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1446/2913] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:13 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:11:14.053517 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d973255-b2a1-49e8-aa68-5cf56b806fa5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.064290 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d973255-b2a1-49e8-aa68-5cf56b806fa5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.120369 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.121086 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.121367 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.121644 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.122305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.124234 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.144381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1468/2914] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:14 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:14.148338 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.159197 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99bfc-2d23-43c7-b5df-55aa704f9a87 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.163750 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.164351 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.164663 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.164939 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.165616 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.167187 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.168402 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:14.180335 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1447/2915] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:14.183972 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.194105 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526feea-20cd-459f-8180-76204f615bc2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.199028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadbea0-cbe2-4373-a6df-4aaf76e1da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.199588 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbea0-cbe2-4373-a6df-4aaf76e1da80 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.199855 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbea0-cbe2-4373-a6df-4aaf76e1da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.200135 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbea0-cbe2-4373-a6df-4aaf76e1da80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.200849 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadbea0-cbe2-4373-a6df-4aaf76e1da80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.202979 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadbea0-cbe2-4373-a6df-4aaf76e1da80 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.212756 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1469/2916] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:14.217745 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.227682 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1bef6-debd-439a-bccc-ef870cbca2ef admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.232320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.233022 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.233312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.233589 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] PATH_INFO: `/v3/projects/fb69b722357341f8b08320f5008178d4/users/7ee0dc0318fd4fc8a712ab43a3063423/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.247870 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb69b722357341f8b08320f5008178d4, user_id=7ee0dc0318fd4fc8a712ab43a3063423, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.250666 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.263018 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1448/2917] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:14 2022] PUT /identity/v3/projects/fb69b722357341f8b08320f5008178d4/users/7ee0dc0318fd4fc8a712ab43a3063423/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:14.273835 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298c362-d1a4-4d82-8282-ef10fbe187ee None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.274121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298c362-d1a4-4d82-8282-ef10fbe187ee None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.274400 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298c362-d1a4-4d82-8282-ef10fbe187ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.282291 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:14.305686 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a298c362-d1a4-4d82-8282-ef10fbe187ee None None] MFA Rules not processed for user `7ee0dc0318fd4fc8a712ab43a3063423`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:14.306881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a298c362-d1a4-4d82-8282-ef10fbe187ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:14.369101 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1470/2918] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:14.374163 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.434337 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c24279-a166-499c-aca4-b61df728bab1 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.439708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.440324 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.440639 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.440922 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] PATH_INFO: `/v3/projects` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.441591 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.443481 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.470016 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1449/2919] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:14 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:11:14.473911 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298c362-d1a4-4d82-8282-ef10fbe187ee None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.485317 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298c362-d1a4-4d82-8282-ef10fbe187ee None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.541712 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.542376 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.542657 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.542932 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.543614 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.546256 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.569278 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1471/2920] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:14 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:14.573263 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.583775 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d1e96-a065-457b-a404-c9d69ba16953 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.588607 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.589216 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.589499 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.589767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.590443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.592649 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.603813 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1450/2921] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:14.608639 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.618760 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97cdf1-b2d6-4d60-b8f4-a2f040934a36 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.623443 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.624529 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.624840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.625117 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] PATH_INFO: `/v3/projects/305565a258314249a6c5f23e00f88b0f/users/5792a98cd6054fb591ba831f3a2dfe59/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.639664 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305565a258314249a6c5f23e00f88b0f, user_id=5792a98cd6054fb591ba831f3a2dfe59, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.642411 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.656245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1472/2922] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:14 2022] PUT /identity/v3/projects/305565a258314249a6c5f23e00f88b0f/users/5792a98cd6054fb591ba831f3a2dfe59/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:14.660213 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.670032 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91daa5b-bb71-426c-b096-07942947e31f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.727629 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4357d8-e82c-4c0e-bc51-2bc9329e8f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.728183 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4357d8-e82c-4c0e-bc51-2bc9329e8f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.728465 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4357d8-e82c-4c0e-bc51-2bc9329e8f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.728767 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4357d8-e82c-4c0e-bc51-2bc9329e8f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.729460 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4357d8-e82c-4c0e-bc51-2bc9329e8f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.731043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4357d8-e82c-4c0e-bc51-2bc9329e8f07 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.741681 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1451/2923] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:14.745845 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.756213 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c686a-7483-4007-ac88-ae49004274f4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.760898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.761555 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.761840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.762121 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] PATH_INFO: `/v3/projects/305565a258314249a6c5f23e00f88b0f/users/5792a98cd6054fb591ba831f3a2dfe59/roles/8b01a7ce4ad14b1aac5e3b46f48a967d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.764092 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] RBAC: Authorizing `identity:create_grant(project_id=305565a258314249a6c5f23e00f88b0f, user_id=5792a98cd6054fb591ba831f3a2dfe59, role_id=8b01a7ce4ad14b1aac5e3b46f48a967d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:14.766730 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:14.776720 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1473/2924] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:14 2022] PUT /identity/v3/projects/305565a258314249a6c5f23e00f88b0f/users/5792a98cd6054fb591ba831f3a2dfe59/roles/8b01a7ce4ad14b1aac5e3b46f48a967d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:14.788262 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.788762 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.788844 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:14.817601 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] MFA Rules not processed for user `5792a98cd6054fb591ba831f3a2dfe59`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:14.818798 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:14.848486 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:14.887004 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1452/2925] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:14.898727 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:14.900452 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:14.946994 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:14.959384 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d2f7e-4727-4ae2-af10-2c64c529538a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:14.963044 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:14.963898 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:14.964179 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:14.964451 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:15.020671 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:15.022028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:15.031861 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1474/2926] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:16.521320 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:16.522707 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:16.535441 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4817085-596f-4757-9b25-5345a64d0c9a None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:16.539058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:16.539674 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:16.539960 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:16.540242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:16.596991 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:16.598421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:16.607325 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1453/2927] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:17.141318 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:17.150006 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2081c9a-ac15-4e30-b385-81e45f9078a3 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:17.153660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3e527-c528-4519-96b5-ab2ccf9fa57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:17.154333 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e527-c528-4519-96b5-ab2ccf9fa57c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:17.154628 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e527-c528-4519-96b5-ab2ccf9fa57c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:17.155580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e527-c528-4519-96b5-ab2ccf9fa57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:17.164586 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3e527-c528-4519-96b5-ab2ccf9fa57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:17.166015 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3e527-c528-4519-96b5-ab2ccf9fa57c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:17.175512 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1475/2928] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:11:21.028503 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:21.090985 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba3e1a-9e65-490d-9085-d964ed27521d service cinder] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:21.095654 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:21.096298 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:21.096617 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:21.096899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] PATH_INFO: `/v3/users/9b9429309b2b4ae482798bfb6f21a193` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:21.098058 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9429309b2b4ae482798bfb6f21a193)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:21.099974 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:21.145476 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9429309b2b4ae482798bfb6f21a193'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.146126 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9429309b2b4ae482798bfb6f21a193'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.158548 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9429309b2b4ae482798bfb6f21a193'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.160531 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:21.166827 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9429309b2b4ae482798bfb6f21a193'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.167461 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9429309b2b4ae482798bfb6f21a193'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.179735 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1454/2929] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:21 2022] DELETE /identity/v3/users/9b9429309b2b4ae482798bfb6f21a193 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:21.184277 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:21.184779 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:21.184881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:21.204351 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:21.205572 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:21.263189 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1476/2930] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:21.273202 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:21.274588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:21.334586 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f75334-778b-4e19-a8e8-c7b35c16e1f5 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:21.338574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:21.339201 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:21.339485 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:21.339765 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:21.349452 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:21.350882 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:21.360479 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1455/2931] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:11:21.625455 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:21.626955 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:21.639028 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7a34f-f72d-442f-97fd-07f8707147ce None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:21.644626 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:21.645280 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:21.645569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:21.645853 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] PATH_INFO: `/v3/projects/68dd61cf13794ccc91c3c86c686b79f7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:21.647017 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68dd61cf13794ccc91c3c86c686b79f7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:21.648991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:21.694838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68dd61cf13794ccc91c3c86c686b79f7', 'request_id': 'req-5bb18578-1902-49e1-8e11-13bdc35c9690'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.701490 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68dd61cf13794ccc91c3c86c686b79f7', 'request_id': 'req-5bb18578-1902-49e1-8e11-13bdc35c9690'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.707881 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68dd61cf13794ccc91c3c86c686b79f7', 'request_id': 'req-5bb18578-1902-49e1-8e11-13bdc35c9690'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.718952 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] The token cache is being invalidate because project 68dd61cf13794ccc91c3c86c686b79f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:21.719243 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.719904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.721445 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1477/2932] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:21 2022] DELETE /identity/v3/projects/68dd61cf13794ccc91c3c86c686b79f7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:21.725686 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:21.735637 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf07ee-6a56-4176-9f9e-da91e7cc8470 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:21.791398 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:21.792041 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:21.792340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:21.792646 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] PATH_INFO: `/v3/users/7ee0dc0318fd4fc8a712ab43a3063423` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:21.793828 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee0dc0318fd4fc8a712ab43a3063423)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:21.795899 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:21.839780 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee0dc0318fd4fc8a712ab43a3063423'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.840431 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee0dc0318fd4fc8a712ab43a3063423'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.852900 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee0dc0318fd4fc8a712ab43a3063423'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.859823 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee0dc0318fd4fc8a712ab43a3063423'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.860457 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee0dc0318fd4fc8a712ab43a3063423'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:21.872942 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1456/2933] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:21 2022] DELETE /identity/v3/users/7ee0dc0318fd4fc8a712ab43a3063423 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:22.152661 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:22.154113 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:22.167286 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb18578-1902-49e1-8e11-13bdc35c9690 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:22.172880 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:22.229354 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:22.230001 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:22.230274 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:22.230554 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] PATH_INFO: `/v3/projects/fb69b722357341f8b08320f5008178d4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:22.231751 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb69b722357341f8b08320f5008178d4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:22.233740 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:22.281003 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb69b722357341f8b08320f5008178d4', 'request_id': 'req-94f02373-c2b1-4c3a-b421-f352c234c455'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.288123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb69b722357341f8b08320f5008178d4', 'request_id': 'req-94f02373-c2b1-4c3a-b421-f352c234c455'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.294263 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb69b722357341f8b08320f5008178d4', 'request_id': 'req-94f02373-c2b1-4c3a-b421-f352c234c455'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.305341 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] The token cache is being invalidate because project fb69b722357341f8b08320f5008178d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:22.305630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.306265 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.307742 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1478/2934] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:22 2022] DELETE /identity/v3/projects/fb69b722357341f8b08320f5008178d4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:22.312584 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:22.313986 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:22.326565 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cccd2e-bd13-45b7-8ff9-97b6decce70f admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:22.331436 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:22.387969 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:22.388920 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:22.389811 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:22.390093 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] PATH_INFO: `/v3/users/5792a98cd6054fb591ba831f3a2dfe59` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:22.391250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5792a98cd6054fb591ba831f3a2dfe59)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:22.393186 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:22.440426 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5792a98cd6054fb591ba831f3a2dfe59'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.441116 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5792a98cd6054fb591ba831f3a2dfe59'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.453534 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5792a98cd6054fb591ba831f3a2dfe59'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.460424 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5792a98cd6054fb591ba831f3a2dfe59'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.461110 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-5f298c5e-6cee-460e-8f2e-71ef190703d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5792a98cd6054fb591ba831f3a2dfe59'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.476157 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1457/2935] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:22 2022] DELETE /identity/v3/users/5792a98cd6054fb591ba831f3a2dfe59 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:22.755991 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:22.757580 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:22.771591 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f02373-c2b1-4c3a-b421-f352c234c455 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:22.830902 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:22.831569 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:22.831846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:22.832116 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] PATH_INFO: `/v3/projects/305565a258314249a6c5f23e00f88b0f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:22.833288 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305565a258314249a6c5f23e00f88b0f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:22.835365 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:22.877176 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:22.884838 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305565a258314249a6c5f23e00f88b0f', 'request_id': 'req-c85bb36e-bd26-4ae7-b815-21f1a0cef168'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.892793 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305565a258314249a6c5f23e00f88b0f', 'request_id': 'req-c85bb36e-bd26-4ae7-b815-21f1a0cef168'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.898947 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305565a258314249a6c5f23e00f88b0f', 'request_id': 'req-c85bb36e-bd26-4ae7-b815-21f1a0cef168'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.910459 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] The token cache is being invalidate because project 305565a258314249a6c5f23e00f88b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:22.910750 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.911381 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c85bb36e-bd26-4ae7-b815-21f1a0cef168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:22.912910 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1479/2936] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:22 2022] DELETE /identity/v3/projects/305565a258314249a6c5f23e00f88b0f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:22.918952 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a482700-b55b-4c1a-9d40-740091153ec5 None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:22.919250 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a482700-b55b-4c1a-9d40-740091153ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:22.919561 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a482700-b55b-4c1a-9d40-740091153ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:22.941033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-5a482700-b55b-4c1a-9d40-740091153ec5 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:22.942198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-5a482700-b55b-4c1a-9d40-740091153ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:23.001082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1458/2937] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:23.009437 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729164-cfbf-4f98-823d-d49d39abad61 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.009722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729164-cfbf-4f98-823d-d49d39abad61 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.010000 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729164-cfbf-4f98-823d-d49d39abad61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.029630 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-60729164-cfbf-4f98-823d-d49d39abad61 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:23.031495 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-60729164-cfbf-4f98-823d-d49d39abad61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:23.039486 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1480/2938] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:11:23.043708 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a482700-b55b-4c1a-9d40-740091153ec5 None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:23.054033 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a482700-b55b-4c1a-9d40-740091153ec5 None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:23.058675 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:23.059290 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.059574 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.059852 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] PATH_INFO: `/v3/domains` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.060588 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80162) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 07:11:23.061065 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:23.062673 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:23.074670 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1459/2939] 10.222.0.33 () {58 vars in 1153 bytes} [Fri Jul 29 07:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 07:11:23.078958 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60729164-cfbf-4f98-823d-d49d39abad61 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:23.088697 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60729164-cfbf-4f98-823d-d49d39abad61 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:23.093949 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:23.094568 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.094848 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.095123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.095787 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:23.097722 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:23.125308 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1481/2940] 10.222.0.33 () {60 vars in 1153 bytes} [Fri Jul 29 07:11:23 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 07:11:23.129518 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:23.139599 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b71b-dae6-4dd1-a35f-1a21cb79e156 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:23.148340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:23.199692 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:23.200242 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.200528 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.200838 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] PATH_INFO: `/v3/users` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.201502 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:23.203800 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:23.224730 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1460/2941] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 29 07:11:23 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 07:11:23.228916 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:23.240840 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc338cc2-ee3b-4e0e-92bf-c76dc0e8a606 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:23.245687 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:23.246305 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.246595 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.246904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.247597 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:23.249271 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:23.259147 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1482/2942] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:23.262981 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:23.273443 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b58d0-d3ae-4f6e-bba6-ea556c59fac9 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:23.278043 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe5770-d54c-4829-b068-0c5e7bef79c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:23.278648 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5770-d54c-4829-b068-0c5e7bef79c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.278934 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5770-d54c-4829-b068-0c5e7bef79c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.279211 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5770-d54c-4829-b068-0c5e7bef79c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.280506 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe5770-d54c-4829-b068-0c5e7bef79c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:23.282119 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe5770-d54c-4829-b068-0c5e7bef79c0 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:23.292741 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1461/2943] 10.222.0.33 () {58 vars in 1122 bytes} [Fri Jul 29 07:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 07:11:23.296978 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:23.306874 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b4c4e-ac69-4991-8696-0a151daccd1c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:23.311857 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:23.312520 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.312834 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.313118 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] PATH_INFO: `/v3/projects/81280c44a59244bdb1e5e54f1a7f15a2/users/60709e405f194425ac1333d9f65eaadb/roles/5f97bcaf49fe487d8def3bf0991dcf15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.328195 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81280c44a59244bdb1e5e54f1a7f15a2, user_id=60709e405f194425ac1333d9f65eaadb, role_id=5f97bcaf49fe487d8def3bf0991dcf15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:23.330885 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:23.344219 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1483/2944] 10.222.0.33 () {60 vars in 1483 bytes} [Fri Jul 29 07:11:23 2022] PUT /identity/v3/projects/81280c44a59244bdb1e5e54f1a7f15a2/users/60709e405f194425ac1333d9f65eaadb/roles/5f97bcaf49fe487d8def3bf0991dcf15 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:23.354519 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc35f2-d95f-4dca-a936-7d52ca1c732e None None] REQUEST_METHOD: `POST` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.354988 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc35f2-d95f-4dca-a936-7d52ca1c732e None None] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.355082 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc35f2-d95f-4dca-a936-7d52ca1c732e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.382130 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.auth.core [None req-c9dc35f2-d95f-4dca-a936-7d52ca1c732e None None] MFA Rules not processed for user `60709e405f194425ac1333d9f65eaadb`. Rule list: `[]` (Enabled: `True`). {{(pid=80162) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:23.383365 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c9dc35f2-d95f-4dca-a936-7d52ca1c732e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:23.446312 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1462/2945] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 07:11:23.458123 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:23.459801 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:23.517043 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678738ac-af48-4c46-9cad-40bfffd4bf52 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:23.520754 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db04575-b8b6-43fe-a0ce-44a843b7e2ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1feaf015a847ffa719fd3a53de02e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:23.521374 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db04575-b8b6-43fe-a0ce-44a843b7e2ef service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:23.521660 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db04575-b8b6-43fe-a0ce-44a843b7e2ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:23.521941 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db04575-b8b6-43fe-a0ce-44a843b7e2ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:23.531025 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db04575-b8b6-43fe-a0ce-44a843b7e2ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:23.532449 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db04575-b8b6-43fe-a0ce-44a843b7e2ef service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:23.541322 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1484/2946] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 07:11:26.088731 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc35f2-d95f-4dca-a936-7d52ca1c732e None None] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:26.169193 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc35f2-d95f-4dca-a936-7d52ca1c732e None None] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:26.174308 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:26.174940 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:26.175220 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:26.175497 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] PATH_INFO: `/v3/users/60709e405f194425ac1333d9f65eaadb` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:26.176677 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60709e405f194425ac1333d9f65eaadb)` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:26.178572 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:26.182947 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:26.225073 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60709e405f194425ac1333d9f65eaadb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.225713 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60709e405f194425ac1333d9f65eaadb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.237976 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60709e405f194425ac1333d9f65eaadb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.244705 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60709e405f194425ac1333d9f65eaadb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.245340 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.notifications [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60709e405f194425ac1333d9f65eaadb'} {{(pid=80162) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.259099 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1463/2947] 10.222.0.33 () {58 vars in 1224 bytes} [Fri Jul 29 07:11:26 2022] DELETE /identity/v3/users/60709e405f194425ac1333d9f65eaadb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:26.263653 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:26.264700 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:26.264816 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:26.294878 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] MFA Rules not processed for user `97726e6afd31490a96b8f997adddff27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 07:11:26.296645 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:26.346521 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:26.360396 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1485/2948] 10.222.0.33 () {58 vars in 958 bytes} [Fri Jul 29 07:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 07:11:26.367141 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:26.368580 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:26.425771 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed315a3-ad57-4638-a05a-a26c79cb37f4 admin admin] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:26.429421 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edf0a00538cf4e10b3bf266f48ea8cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:26.430038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:26.430321 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:26.430598 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:26.440029 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:26.441540 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:26.450867 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1464/2949] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:11:26.715585 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:11:26.717056 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:11:26.737232 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2e164-c5c1-40e4-bf9b-8b5eab36c809 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:11:26.741766 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97726e6afd31490a96b8f997adddff27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc5535113d8e4690869b5d23e1b0a8b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:11:26.742413 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:11:26.742753 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:11:26.743033 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] PATH_INFO: `/v3/projects/81280c44a59244bdb1e5e54f1a7f15a2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:11:26.744173 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81280c44a59244bdb1e5e54f1a7f15a2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:11:26.746108 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:11:26.791681 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81280c44a59244bdb1e5e54f1a7f15a2', 'request_id': 'req-7bc32ed3-0b4e-4044-bb15-65af461d2679'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.799936 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81280c44a59244bdb1e5e54f1a7f15a2', 'request_id': 'req-7bc32ed3-0b4e-4044-bb15-65af461d2679'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.806543 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81280c44a59244bdb1e5e54f1a7f15a2', 'request_id': 'req-7bc32ed3-0b4e-4044-bb15-65af461d2679'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.817846 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] The token cache is being invalidate because project 81280c44a59244bdb1e5e54f1a7f15a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 07:11:26.818129 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.818761 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bc32ed3-0b4e-4044-bb15-65af461d2679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 07:11:26.820245 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1486/2950] 10.222.0.33 () {58 vars in 1233 bytes} [Fri Jul 29 07:11:26 2022] DELETE /identity/v3/projects/81280c44a59244bdb1e5e54f1a7f15a2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 07:11:36.446711 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:11:36.814904 dv-cinder-834822-7-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 07:15:19.719825 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] Authenticating user token {{(pid=80162) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 07:15:19.721557 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:15:19.854841 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a6999-0bde-4846-9a85-ceb9997613c3 service neutron] Validating token access rules against request {{(pid=80162) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 07:15:19.911824 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743ca22-2bdf-4298-bfe7-3bfe9d06b6cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4986a5d3e6c4adf832c1b1ec8dd768a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b42836f77f104b648efac765cddf6b93', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80162) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 07:15:19.912435 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743ca22-2bdf-4298-bfe7-3bfe9d06b6cb service placement] REQUEST_METHOD: `GET` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 07:15:19.912761 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743ca22-2bdf-4298-bfe7-3bfe9d06b6cb service placement] SCRIPT_NAME: `/identity` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 07:15:19.913038 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743ca22-2bdf-4298-bfe7-3bfe9d06b6cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80162) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 07:15:19.915521 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.fernet_utils [None req-a743ca22-2bdf-4298-bfe7-3bfe9d06b6cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80162) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 07:15:19.926433 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743ca22-2bdf-4298-bfe7-3bfe9d06b6cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 07:15:19.927995 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743ca22-2bdf-4298-bfe7-3bfe9d06b6cb service placement] RBAC: Authorization granted {{(pid=80162) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 07:15:19.938538 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: [pid: 80162|app: 0|req: 1465/2951] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 29 07:15:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 07:15:29.934198 dv-cinder-834822-7-iscsi devstack@keystone.service[80162]: DEBUG dbcounter [-] [80162] Writing DB stats keystone:SELECT=84 {{(pid=80162) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}